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This study explores the use of the adjunct question 
paradigm in self-instruction and distance education materials. The 
subjects were 187 high school students who read text passages and 
answered one high level (implication or inference) or low levol 
(verbatim or paraphrase) question. These questions were either 
inserted in the text or massed at the end of the passage. Subjects 
were either allowed to look back at the text while answering the 
questions (the freedom group, FREE) or were not allowed to look back 
(the read-read-quf. St ion group, RRQ; the read-question-read group, 
RQR). One control group read the passages twice without questions and 
another neither read the passages nor answered the questions. All 
subjects were tested on recall of low level and performance on high 
level posttest items. On high level incidental items, the RQR group 
performed better than che RRQ group, but the FREE group recalled more 
low level incidental material than the RQR group. In the two 
no-lookback groups, those who were asked high level adjunct questions 
performed better when the questions were massed together, while those 
who were asked low level adjunct questions performed better when the 
questions were inserted. In general the lookback group recalled as 
much, and performed as well, as the two no-lookback groups. For all 
incidental dependent variables, most treatment groups did not score 
significantly higher than the READ-TWICE control group. Discussions 
of the limitations of the study that may have influenced the results 
and the implications of the findings conclu^" the report. A reference 
list of 101 items is appended. (Author/RP) 
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ABSTRACT 



Senior high school students read text passages and answered on-; high level 
(implication or inference) or low level (verbatim or paraphrase) question, either 
inserted in the text or massed at the end ot the passage. Subjects either were 
allowed to look back at the test while answering the questions (the freedom 
group, FREE); or were not allowed to look back (the read-read-question group, 
RRQ; the read-question-read group, RQR). One control group read the passages 
without questions and another neither read the passages nor answered the 
questions. All subjects were tested on recall of low level and performance on 
high level posttest items related to intentional material (related to adjunct 
question) and incidental material (unrelated to adjunct question). 

On high level intentional V-ms, the RQR group performed better than the RRQ 
group. The FREE group recalled more low level incidental material than the RQR 
group. The group asked high level adjunct questions performed better when the 
questions were massed together while the group asked low level adjunct 
questions performed better when the questions were inserted. This interaction 
occurred for the two "no-Iookback" groups (RRQ, RQR) but not for the lookback 
group (FREE). In general the lookback group recalled as much, and performed as 
well, as the two no-lookhack grc >ps. 
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THE PROBLEM 

Students who study at institutions of higher education receive a large amount of 
their instruction from the texts they read after attending an organizing lecture. 
Some receive instruction from self-instructional materials. Students who study for 
degrees at c dista ice, typically receive, through the postal service, learning 
packages which contain printed textual material, printed study guides, 
audio-cassettes and assignments or exercises. Of these, it is the printed texts 
and self-instructional study guides that contain most of the material to be learned. 
In mixed-mode institutions, distance learners take the same examinations and have 
the same assignment deadlines as the fulltime students who study at the same 
university. If distance students are to compete adequately with fulltime students, 
they must receive instructional materials of high quality. 

Many of the instructional prescriptions developed by Gagne (1985), Gagne and Briggs 
<i979). Cropper (1983), Landa (1983), Merrill (1983), Reigeluth (1V83), Scandura (1983) 
and other instructional theorists may be adapted for use with self-instructional or 
distance learning materials. The instructional prescriptions that seem most easily 
adapted are those prescriptions related tc the teaching of concepts, procedures and 
principles (see Merrill, 1983), Gagne's nine events of instruction (see Gagne, 1985) 
and those related to motivation (see Keller 1983). 

However, most self -instructional packages, distance materials and textbooks are not 
well designed. Often, they do not attempt to teacn the application of concepts, 
theories and procedures; rother they provide a lot of verbal information from which 
the learner must extract that which may be useful. Apart from Gagne's nine events 
of instruction (Gagne, 1985; Gagne and Briggs, 1979), which are vrry Icose 
prescriptions, there are no theoretical prescriptior^s about how to help learners 
obtain a meaningrul understanding of verbal information presented in print. 

The research on distance education (Holmberg, 1981 and 1982) does not address the 
problem of how to make the learning of verbal information more meaningful for the 
self-learner - the researchers seem to be more concerned with high technology 
delivery systems. Two recent attempts to develop theories of distance education 
(Perraton, 1981; and Holmberg, 1985) do not address the problem. 
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Benjamin Bloom, in his model of school learning (1976), identifies "quality of 
Instruction" as an important variable in learning. Quality or instruction, measured 
during instruction, is a good predictor of achievement meascired after instruction - 
in his synthesis Bloom found that typically r « 0.50. The most important component 
of quality of instruction is "participation" - the degree to which the students are 
"actively involved in their learning" (p. 134). 

The participation of the learner in the learning task relates to the active 
involvement of the learner in carrying out the directions s-ated as cues. An active 
rather than passive involvement in learning is required. Individual students differ 
in the amount of participation in the learning process they both desire and require 
and in the amount of practice they need before they gain proficiency (Atkinson, 1968; 
Glaser, 1968). However, as students move through the various tasks in a mastery 
learning situation, the difference in amount of participation and practice needed 
becomes less (r'.oom, 1974; Anderson, 1984). 

When achievement is related to group participation, the correlation is usually only r 
» >0.27 (Morsh, 1956; Lysakowski and Walberg, 1982; Walberg, 1984), but because 
students differ in the amount of participation they need, individual participation is 
more highly related to achievement than is group participation. Twenty percent of 
the variance (r » +0.45 approx.) in individual achievement is accounted for by 
individual variance in participation (Anderson, 1976; Sjogren, 1967; Siegel, 1963; 
Krauskopf, 1963; Olson, 1931; Edminston and Rhoades, 1959; Walberg, 1984). 

Bloom shows (1976, P. 134) that the extent, type and quality of participation in the 
learning process is the best single indicator of quality of instruction. If students 
are participating fully, the cues, reinforcement and feedback must be adequate. 

One of the most effective and easiest ways for developers to increase the quality of 
instruction is for them to increase the degree of student participation. Normally, 
in distance materials developed to teach concepts, principles and theories, faculty 
can Increase participation by increasing the number of questions, activities, 
problems, case studies, and vocationally related projects. However, what should be 
done when the learners are attempting to learn from the continuous prose in less 
well designed self-instructional or distance materials, or from the prose in their 
required texts. 
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As distance and self-learners typically receive verbal information in prose form, 
they must, in Sloom's terms, "participate in the learning" by interacting with the 
prose, if they are to obtain a meaningful understanding of the verbal information. 
One way to increase participation, is to provide questions and activities within the 
prose materials. Perhaps it is from reading research that insights can be obtained 
into how to make verbal information more meaningful for self-learners. One likely 
source of inspiration is the research into the effectiveness of adijnct questions - 
qu<!Stions inserted or embedded in the text. 

One of the primary modes through which students acquire information and knowledge 
in an academic setting, and particularly in distance education or self-study, is by 
reading expository prose. There have been numerous studies aimed at evaluating 
the effectiveness of adjunct questions in a prose learning setting. (See Anderson 
and Biddle, 1975; Andre, 1979; Faw and Waxier, 1976; Hamaker, 1986; Hamilton, 
1985a; Rickards, 1979 for reviews of the research.) In general, adXinct questions 
have been found to be effective. Can the knowledge obtained by adXinct question 
researchers be used by instructional designers developing strategies for use in 
distance education or in mediated self-instruction? 

In his analysis of distance education with respect to contemporary teaching models, 
Baath (1979) argues that Rothkopf's model for written instruction seems to be 
directly applicable to print based distance education. However, the differenc'js 
between the experimental adjunct question studies (where subjects may not look at 
the text while answering the questions) and real life distance or self-study 
education (where students are free to look if they wish) are so great that to 
generalize findings from the studies to distance education and self-instruction may 
not be prudent. It may be concluded, however, from Baath 's analysis, that the 
development of adjunct study guides to textual material may be a useful addition to 
the instructional repertoire of the distance educator and the developer of 
self-instructional materials. 

Because adjunct question research Is difficult to generalize to the real world, and 
because distance education students typically behave very differently from subjects 
in adjunct question research (see the following review of the studies), then 
prescriptions for the use of adjunct questions in distance education and 
self-instruction need to be developed and tested. The aim of this study is to 
determine the applicability of adjunct question research findings to the development 
of such prescriptions. 
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The Adjunct QugsMon Research Paradigm 

Early Research - A Behavioral Perspective. 

During the last twenty years there has been considerable interest in the use of 
inserted or adjunct questions designed to help students learn from textual material. 
Rothkopf (1965) stimulated the current interest when he investigated the 
"mathemagenic** (birth of learning) effects of adjunct questions and developed a 
research paradigm similar to the one described below. 

Typically, in adjunct question research, adjunct questions are placed either before 
or after the information in the text needed to answer them. They are intended to 
increase the likelihood that readers will correctly answer criterion test questions. 
Although variations exist, the text segments and questions are usually provided on 
separate sheets. The reader is neither allowed to turn back to a previous page once 
he has turned it, nor allowed to take notes while reading. 

In most studies, a posttest assesses the effects of the adjunct questions and the 
results compared to those of a "read oniy, no questions" control group. The amount 
of the questioned material recelled, the intentional learning, and the amount 
recalled of material for which there was no rjLiestion, the incidental learning, seei t 
to be of interest. Rothkopf (1965) argues that the mathemagenic effect, the amount 
of incidental material recalled, is of greater educational significance. 

The procedures described above are not very similar to the way in which typical 
distance students interact with their print-based instructional materials. Distance 
students and self-learners would typically look back over a passage while answering 
a question. Also, the faculty involved in providing distance education are wore 
interested in the intentional learning than the incidental learning - although the 
learning of incidental material would be advantageous to students. For these 
reasons, the results of adjunct question research must be carefully analyzed before 
the implications of their use in distance education, and in the development of 
self-instructionel materials, can be clarified. 
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More Recent Research - A More Cognitive Perspective 

Since Rothkopf's original neo-behavioral perspective, that the questions act as 
reinforcing stimuli, there has been a shift towards a cognitive perspective, which 
emphasizes the active role of the learner. The questions are no longer thought of 
as merely assisting in th^ retrieval of information from memory but are considered 
as assisting in the storage of the information. (Many of the most influential 
Instructional design theorists sesm yet to make this transition.) 

There has also been a shift of interest from the effects of factual or verbatim level 
questions to those of a higher or semantic level. Some of the recent reviews of 
adjunct question research (Andre, 1979; Carrier and Fautsch-Patridge, 1981; and 
Rickards, 19?'.* have really been attempts to examine the nature of the semantic 
processing activities generated in the reader by the adjjnct questions. 

In recent adjunct question research, one of the independent variables (the 
treatments) Is typically the level of processing Induced In the reader by the ad^nct 
question, a second Is the position (pre- or post-) of the question with respect to the 
passage, and tnother is the frequency of presentation of the questions or, as It Is 
often presented, the amount of prose between the Inserted questions. The reviews 
(Hamilton, 1985; Hamaker, 1986) Imply that higher level processing typically leads to 
greater recall and greater use of the Information presented, as does more frequent 
questioning and the use of post rather than prequestions. 

The measured learning outcome (the dependent variable) is usually the level of 
processing required by the criterion posttest and, often, whether the posttest Item 
is related to Intentional or Incidental material. Inference and Implication posttest 
Items are hypothesized to require a higher level of semantic processing than do 
verbatim and paraphrase questions. Incidental learning Is considered more 
important than Intentional learning (see previous comment). 

Except for Andre's model (1979) which does not apply to all of the variables studied, 
the reviews (Hamilton, 1985; Hamaker, 1986; and Rlckards, 1979) Indicate that there 
does not appear to be a solid theoretical foundation for adjunct question research - 
that Is, there Is no sound explanation for the results; nor does the present paradigm 
approximate any real learning situation, and certainly not distance education or 
self-Instruction. However, there has been a large number of studies and several 
tentative conclusions have been drawn. Perhaps these studies offer some Insight 
for the developer of self-instruction and for the distance educator, 
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Review of Adkjnct Ques tion Research 

There have been several recent, comprehensive reviews of the effects of adjunct 
questions on learning from text (Andre, 1979; Anderson and Biddle, 1975; Carrier 
and Fatsch-Patridge, 1981; Duchastel, 1983; Faw and Waller, 1976; Hamaker, 1986; 
Hamilton, 1985; Rickards and Denner, 1978, Rickards and Denner, 1979; Rickards, 
1979; van Hout-Halters, 1980)* The reviews show that disagreements exist about 
how adjunct questions affect the processing of textual information. 

Rickards and Denner (1978), in their review of the development of adjunct question 
research, argue that the critical factors in {5uch research are: 

* the variables examined (position, type, frequency, level), 

« the interactions among these variables, 

» the processes induced in the reader by the various questions, 

» and individual diffijrences in cognitive processing associated with 
adjunct questions. 

Type of Questions 

A large number of experiments have found that verbatim postquestions produce 
better performance on verbatim criterion tests than do no questions at all 
(Anderson and Biddle, 1975; McGaw and Grotelueschen, 1972; Boker, 1972; Reynolds 
et al., 1979; Rickards et al., 1979; Rothkopf, 1966; Rothkopf and Bisbicos, 1967; 
Sagria and Di Vesta, 1978; Shavelson et ah, 1974). 

However, Watts and Anderson (1971) and Friedman and Rickards (1981) found that 
the no-question control group performed as well as the experimental group on 
verbatim posttest items. Hamilton (1985) argues that the Watts and Anderson 
results can be explained by the ease with which the adjunct questions could be 
answered (99X correct), while the adjunct questions used by Friedman and Rickards 
were very difficult (36% correct) and provided no semantic cues to the learner (they 
were four versions of the same sentence with the position of one word altered - see 
Friedman and Rickards, 1981, p 429). Hamilton's analysis also demonstrates the 
need to report performance on the adjunct questions themselves in any study of the 
effectiveness of adjunct questions in prose. 
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Hamilton (1985) and Hamaker (1986) conclude that verbatim post and prequestions 
consistently increase the probability o£ recall of intentional material - material 
related to the adjunct question. In fact, for the retention of intentional verbatim 
material, the position of the adjunct question seems to be unimportant. However , 
only verbatim postquestions consistently increase the probability o* recall of 
incidental material - material unrelated to the adjunct question. 

It appears, from this synthesis, that educators should use verbatim postquestions 
in their self-instructional or distance materials if their objectives include the 
verbatim recall of intentional and incidental information. 

Adjunct question researchers have slowly moved from a behavloristic to cognitive 
perspective and now study the level of processing required to answer the adX<nct 
questions. Because rote learning of facts is considered by most present day 
adjunct question researchers to be of little educational value, the earlier studies 
which used verbatim questions are no longer considered significant (Carver , 1972; 
Watts and Anderson, 1971). 

Both the level of processing of questions (verbatim vs. semantic) and the Interaction 
of level and position of question have been Investigated. Generally, higher-level 
(semantic) questions produce more and better organized recall, especially free 
recall, of both Intentional and incidental material than lower-level questions (see 
the reviews of Hamilton, 1985; and Hamaker, 1986). 

Four types of semantic questions have been used; Inference, subsump'lon. 
Implication, and paraphrase questions. Shavelaon et al. (1974) and Andre and 
Womack (1978) found that a group given semantic postquestions performed 
significantly better on Intentional posttest items presented Immediately after the 
passage than a group given verbatim questions and a no-questlon control group. 
Shavelson et al. (1974), Anderson and Biddle ^975) and Friedman and Rlckards (1981) 
found similar results with a delayed test. 

Shavelson et al. (1974) and Rickards (1976) found that inference and implication 
prequestions led readers to correctly answer more intentional implication and 
inference posttest items than did verbatim questions, no questions or questions 
unrelated to the intentional learning. Hamaker's (1986) review shows that higher 
order questions have a much greater "effect size" than verbatim questions on all 
levels of posttest items except repeated verbatim items. 
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Despite the lack of consistent results, there appears to be general agreement that 
higher-level questions are likely to be more effective than lower level as ad>jnct 
aids to text comprehension. This seems to agree with the "levels of processing" 
model iCraik and Lockhart, 1972; Craik and Tulving, 1975; Fisher and Craik, 1977; 
Jacoby and Craik, 1978; Craik, 1979) which argues that higher level adjunct questions 
lead to better processing of both hiyner and lower level material. 

Position and Frequency of Question 

Hamilton (1985) reviews the position of questions (pre or post) studies and 
concludes that postquestions and prequestions increase the probability of recall of 
intentional verbatim material equally. For the retention of intentional semantic 
material, however, semantic pre-questions lead to improved retention. 
Appropriately constructed semantic prequestions can produce a more complete 
processing of the passage as the reader focuses on the "topical interrelationships 
and organization" (Hamilton, 1985, p.77>. Hamaker's review (1986) shows that 
semantic postquestions generally have minimal effects on the processing of 
intentional material. 

Regarding incidental learning, the general conclusion from several reviews 
(Anderson and Biddle, 1975; Andre, 1979; Hamaker, 1986; Hamilton, 1985; Rickarda, 
1979) is that verbatim postquestions lead to better recall of incidental material 
than verbatim prequestions; and while verbatim postquestions lead to better recall 
than read-onJ.y controls, verbatim prequestions do not. Anderson and Biddle (1975) 
in their review suggest that adjunct postquestions facilitate incidental learning 
whereas prequestions inhibit such learning. Semantic pre-questions probably led 
readers to focus on thj intentional material and a so induce a more superficial level 
of processing of incidental material. 

There is no agree nent about the best position for higher-level questions. Felker 
and Dapra (1975) found that semantic postquestions led to better performance on 
both intentional and incidental posttest items than semantic prequestions. 
However, Rickards (1976) arrived at the opposite conclusion. Hamaker in his review, 
and Hamilton in his meta analysis, both agree with Rickards and Mayer (1975) who 
found both positions to be equivalent. 

There is also no agreement about the results of the various frequency of question 
studies. Some studies (see Rickards, 1979) found that as the questions become less 
frequent, the amount of intentional and incidental learning increases for the 

EMC 11 
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prequestlon group but decreases for the postquestion group. Rickards and DiVesta 
(1974) found that while higher level postquestions were more effective when 
occurlng more frequently in text, there was no interaction between frequency and 
position for lower level questions. Andre and Womack <1978) found that 
postquestions placed at the end of each paragraph was preferable to the questions 
being massed at the end of the complete text. 

Rickards and Denner (1978) argue that the reason for the lack of agreement is that 
researchers are unable to define the various levels of semantic processing in 
precise operational terms. In other words, the frequency studies have not used a 
consistent differentiation among levels of questions. Thus, any results about a 
certain level of question having a better effect with a certain frequency of 
presentation has little validity. Also, for the same reason, any interactions 
between frequency and level found also has little vailidity. 

For distance educators, primarily interested in intentional learning, the results of 
these studies indicate that, semantic postquestions will better prepare readers for 
Intentional semantic posttest items than will verbatim postquestions or no 
questions at all. Further^ verbatim prequestions are more useful than verbatim 
postquestions for facilitating the learning of verbatim information. However, if 
the goal is that Incidental learning also occurs then, for both semantic and verbatim 
learning, postquestions are more effective. 

Types of Processing 

Both Andre (1979) and Rickards (1979) have recently attempted to develop models to 
explain th^ type and nature of the cognitive processes adjunct questions produce. 
Rickards (1979, pl93) extended Prase's (1967, 1970) suggestion that adjunct 
questions produce both a backward processing and a forward processing by 
concluding that four processes are probably involved: 

(1) a general backward process involving the mental review of material 
thematically related to that questioned; 

(2) a specific backward process reviewing only the questioned material; 

(3) a general forward process with increased attention to the following 
text; and 

(4) a specific forward process whereby the learner adopts a learninq set 
attuned solely to the prxticular type of information questioned. 
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Prequestions produce a specific forward processing which overfocus the reader's 
attention on a specific tyoe of informetion and depresses incidental learning (see 
the reviews of Hamilton, 1985; and Hamaker, 1986). This effect is greater for 
verbatim than higher order prequestions. Rickards and DiVesta (1974) found that 
factual postquedtions induced a specific backward process while higher level 
postquestions produced a general backwaril process. 

Sefkow and Meyers (1980) studied the backward review effect of both low and high 
order postquestions. The results indicated that a specific backward process is 
used, since memory is enhanced for only the parts of the passages related to the 
post:<uestions. Sefkow and Myers also attempted to dis*,over whether the backward 
processing enhanced storage or retrieval. The results indicated that the effect of 
the post question is to strengthen storage, rather than to merely trigger retrieval. 

Both Reynolds, Stanford, and Anderson (1979) and Reynolds and Anderson (1982) 
found that adjunct postquestions caused a specific forward as well as a specific 
backward effect. Using a computer to present the text, Reynolds et al (1979) found 
that learners spent a disproportionate amount of time on questioned versus 
non-questioned segments of text. Reynolds and Anderso: (1982) found an increase 
in the amount of time spent on text segments related to the content questioned in 
previous postquestions. Subjects spent more and more time connidering each 
question and less and less time in reading the passages (further evidence for the 
notion that: questions enhance storage of information as well as trigger retrieval). 

A problem may occur in self-instruction and distance education if the adXjnct 
questions are toe specific. Because self and di. ance learners have the freedom to 
look back at the pc ssage after reading a postquestion, they may focus too closely on 
the material about which a ques*-'Qn was asked and neglect the other information. 
Howev2r, if Intentional learning is mosre important then this should no^ matter. I£ 
the appropriateness of adjjnct question procedures for self-instruction and 
distance sducation is to be establisiied then this study needs to compare subjects in 
typical adjunct question groups with subjects using typical distance education 
materials. This study m.»,y find that learners in self-learner (or freedom) treatment 
groups recall as much Intentional material as learners in the adjunct question 
groups but recall less of the incidental material. On the other hand, with high^rr 
level questions, the fact that the self-learner has the freedom to read the passage 
both before and after sighting the quesvion may lead to both backward and forward 
general processing and improved learning. 

ERIC 13 
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Levels of Semantic tVocessing 

Perhaps the most important questions that adjunct question research must answer 
are: ••What level of processing is necessary if students are to recall and use 
information in expository prose?" and "What types of questions best facilitate that 
type of processing?" Andre (1979) concluded his lengthy review by arguing that 
while support exists for the notion that higher semantic processing is preferable to 
lower levels of processing, little or no research has compared the various levels of 
semantic processing. Andre goes on to suggest the following as classifications for 
high level semantic questions (p.282): 

(1) application questions that require subjects to choose from among 
various alternatives a new example of a concept or principle 
encountered in text; 

(2) meaningful learning or inference and implication questions requiring 
readers to identify relationships between elements of a passage which 
are implied but not explicitly stated; and 

(3) higher-order questions, defined as being the analysis, synthesis and 
evaluation levels on Bloom's taxonomy. 

Each of Andre's classes is applicable to degree-level self-instructional and 
distance educaJon courses. It is likely, however, that David Merrill's (1983) 
Component Display Theory <CDT) is a better model for teaching the recall and 
application of concepts, principles and procedures than is the use of adjunct 
questions, and that CDT is more easily adapted for self-instruction and distance 
education. This reviewer has not read any studies where classification 3 was used. 
Therefore, since this study is intended to compare typical adjunct question 
procedures with the freedom of self-instruction, it must concentrate on 
classification 2. 

Andre (1979) also classified studies on the basis of the level of posttest employed - 
whether it requires recall of passage material or higher-level functioning. Andre 
argues that, if recall questions are used, we cannot tell if readers are recalling the 
passage rotely or h^ive learned from a passage - thus they may not be particularly 
useful. He also notes J:hat higher-level questions often lead to improved recall of 
facts contained in a passage. If readers have to attend to more of the passage to 
answer the higher level question, then they are able to freely recall more facts than 
those who focus on specific items to answer factual-level questions. Andre calls 
this the directed attention model (DAM). 

J4 
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This model has sparked ar Interesting debate among adijnct question researchers. 
Andre (1979) arguts that if a posttest is only going to ask for the free recall of 
factual material, then improved recall can be achieved in two ways - firstly, by using 
higher order questions to direct attention to more of the passage, and secondly, by 
simply asking more facti^al questions per passage. However, the use of higher level 
questions merely to cause better recall of factual material represents a "clear 
misunderstanding of the instructional use of prose" (p291). Higher level questions 
should help learners develop butter and more organiized recall not merely recall a lot 
of information. 

Rickards (1979), however, arguc*5 that higher level questions not only cause better 
recall but also more organized recall. Higher-level questions affect not only the 
level at which prose is processed but the manner in which it is organized in memory. 
Rickards and DiVesta (1974) and Rickards (1976) show that higher-level questions 
result in more organized and structured recall and in better long-term retention than 
lower-level questions. Simply providing more frequent factual-level questions would 
not achieve similar results. 

Even Andre (1979) argues that his DAM does not fully explain the facilitative effects 
of higher order questions. Different types of questions, by directing the reader's 
attention to particular types of information, ultimately result in inducing different 
processing strategies in the reader. 

There is little agreement among adjunct question researchers about the facilitative 
effects of higher level adjunct questions on posttests containing higher level 
questions. A series of studies conducted by Andre's associates seem to suggest 
that students given factual-level questions actually perform better than those given 
application-level questions. However, Andre (1979, p299) points to the higher 
difficulty level of adjunct application questions and variations in the ability level of 
subjects to explain these results. His general conclusion seems to be. that questions 
may well have iifferent effects on different learners in different situations. 

Rickards and his associates have also studied the level of question problem. 
Friedman and Rickards (1981) presented readers with three different types of 
questions: 

(1) verbatim; 

(2) paraphrase; and 

(3) inferential. 
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They reasoned (p428) that "only inferential questions should (lead readers) to process 
text by relating it to (their) existing knowledge. Sinre (they require) deeper semantic 
processing than that produced by either verbatim or paraphrase questions, (inferential 
questions) should elicit superior learning of questioned and non-questioned material". 

Andre and Sola (1976) suggested that when a reader reads a passage it is encoded 
initially at a shallow level. Thus, any high level postquestion would be too difficult to 
answer and the desired level of processing can not occur. Reading the passage twice, 
before attempting the question, may raise the level of the initial encoding and make 
answering a high level postquestion a little easier by improving the backward 
processing. Andre and Sola also suggested that re-reading the passage after an 
initial attempt to encode it is a better solution to the higher level postquestion 
problem. The high level postquestion inserted between readings of the same passage 
should guide the re-processing of the passage during the second reading and enhance 
forward processing. 

Following the suggestion by Andre and Sola, Friedman and Rickards (1981) required 
subjects either to read a paragraph twice before answering an inserted question (RRQ), 
or to read the paragraph, answer the question, and then re-read the paragraph (RQR). 
They hypothesized that the RQR sequence, due to its potential facilitation of both 
forward ano* backward processing, was superior to the RRQ sequence?. 

The results of ths experiment indicated that both paraphrase and inference questions 
proved superior to verbatim questions for both intentional and incidental semantic and 
verbatim learning. Paraphrase and inference questions also resulted in superior 
performance on intentional items at all levels, including verbatim, implying that a 
transfer of level effect was facilitated by these questions. In addition, subjects 
receiving inference questions outperformed those receiving paraphrase questions in 
both intentional and incidental learning. However, the profound effects of 
presentation sequence suggested by Andre and Sola were not found. Only the RQR 
group exhibited the transfer of level effect, possibly indicating that this sequence 
results in improved learning across cognitive levels. 

Overall, the results of this experiment support the conclusion that "the level of 
semantic processing employed by readers can be manipulated by varying the cognitive 
level of postquestions interspersed in text" (p433). Because of the similarity between 
the procedure followed by RQR subjects and the suggestions about "how to proceed" 
pro 'ided students in many self-instructional and distance courses, it is proposed that, 
in this study, RRQ and RQR groups be compared with "freedom" groups. 
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The greatest limitation of the adjunct question paradigm is its lack of 
generalizability. Real life students are not constrained as are the subjects in 
adjunct question research. Adjunct question subjects may not take notes nor 
underline relevant sections nor re-read the passage nor look back at the passage 
a. .cr reading the question. Students in real life can do all of those things. Often 
the texts they read as part of their courses include questions intendeu to help them 
learn. Studies need to be done to find out how students approach these questions, 
how they should approach these, whether or not the present questions enhance their 
learning, and what the questions should be like if the students learning is to be 
enhanced. 

Very few studies have attempted to explore the use of adjunct question in more 
realistic settings (see Gagne ct al, 1979; Ellis et al, 1980; and Anderson et e\, 1974), 
and none have attempted to determine the effectiveness of adjunct questions in 
study guides to texts, in self-instructional packages and distance education 
materials. Further, few studies have compared typical adjunct question procedure 
with more realistic procedures. It is intended that this study help to fill that void. 

Self-Instruction, Distance Education and the Ad iunct Question Paradigm, 

While reading a text, working through self-instructional packages, and studying 
distance education materials, students are presented with lengthy prose materials 
from which they are expected to learn that which the instructor intended. 
Sometimes the materials are designed to teach the application of a concept, 
procedure or principle, but more often than not the learner is expected to extract 
information from continuous prose. Some distance educators and faculty provide 
study guides that include questions to help the students focus on that which they 
want them to learn and also to help them to process the information at the required 
level. These questions are usually massed at the end of a unit or chapter, but, in 
self-instructional and distance materials are often inserted in the text. 

However, it is important to remember that real-lif^i students can not be forced into 
the artificial procedures of the adjunct question paradigm. They are free to proceed 
as they wish, looking back at the passage while answering the questions if they so 
desire. 
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Shavclson (in Shavelson ct al, 1974b) argued that allowing readers such freedom 
results in a ''fnathemathantic" (death of learning) effect, rather than "mathemagenic" 
(birth of learning) effect because readers focus so much on the material related to 
the adjunct postquestion that they do not process the other information. His view 
is supported by Duchastel (1983) and Duchastel and Nungester (1984) who ascribe the 
lajk of a facilitative effect on incidental posttest questions to the fact that 
students could "lookback**. However, Hamaker (1986), in an extensive review and 
meta-analysis, provides evidence of the faulty logic of these researchers as he 
points out that they did not directly compare lookback and non-lookback groups. 
Hamaker ends up supporting Andre's (1981) view that there is no difference between 
lookback and non-lookback groups. Due to the scarcity of adjunct question research 
studies comparing typical adjunct procedures with those procedures used in 
generallMble real-life settings, and tn^ above described debate, this study should 
compare ••freedom'* groups with typical adjunct question paradigm groups. 

It can be inferred from their practice that some distance educators, and some 
faculty involved in providing self-instructionul materials, disagree with Shavelson 
and believe that including questions in their materials is preferajle to providing no 
questions at all and that higher level questions are preferable to lower level 
questions. However, they have no evidence to support their beliefs and no 
research-based prescriptions about what type of questions to ask their learners. 

As mentioned above, there is also the possibility that good directions (suggesting 
RQR) and well designed higher level questions can lead distance learners to process 
the new material using both backward and forward processing and that this will lead 
to improved learning. With high level implication, inference, analysis, evaluation 
and synthesis questions, readers have to retrieve information they have previously 
stored semantically, to alter that material during semantic rehearsal of the new 
information, and to store the new combination semantically in long-term memory. 

If this is the process that occurs, then allowing distance learners the freedom to 
re-read the study material while attempting to answer high level adjunct questions 
should not lead to the over-focusing some adjunct question researchers suggest. It 
should lead instead to improved cognitive processing and the development of a more 
appropriate cognitive structure. With a more appropriate cognitive structure, the 
learner should better be able to answer intentional and incidental posttest 
questions requiring high level processing, and to have improved retention. 
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If this study is to determine the effectiveness of typical distance education and 
self -instructional procedures with respect to adjunct questions, then the 
effectiveness of such procedures should be compared with the effectiveness of the 
more recent, more logical and more effective adjunct question procedures. Groups 
of learners using s^'lf-study (freedom) versions of the materials should be compared 
with groups of learners using read-question-read versions (RRQ) and 
read-read-question (RQR) versions of the materials after Friedman and Rickards 
(1981). 

Some subjects should receive higher level adjunct postquestion and others lower 
level postquestions. After Andre and Womack (1978), some should receive the 
questions inserted between passages, while others should receive the questions 
massed together at the end of the complete text. The posttest should also include 
high and low level items about both intentional material and incidental material. 
To help with the generalizability of the results to the heterogeneous group of 
learners typically using self-instructional and distance materials, the group of 
subjects in this study should be as heterogeneous as the prose materials will allow. 

Hamaker's (1986) review of the adjunct question research pointed out the "scarcity 
of lookback studies" (p229) and that no studies had yet compared the ad^nct 
question paradigm procedures, where no lookbacks are permitted, with realistic 
procedures where lookbacks are permitted and even encouraged. This study will 
help provide information about the applicability of the results of ad^nct question 
research to generalizable real-life settings such as self-instruction and distance 
education. It will also either confirm that adjunct question researchers have been 
correct in conducting their studies using non-realistic settings or it will show 
adjunct question researchers that they need not persist with such ecologically 
invalid procedures. 
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The Study. 

Independent Variables 

There are three Independent variables: 

(a) The presentation sequence <with three levels) 

* RQR, read the text, attempt the question, re-read the text, 

« RRQ, read the text, read it again, answer the question, 

« FREE, read the text and question as often as needed with 
complete freedom. 

(b) The level oi the adjunct question (with two levels) 

« HI, higher level questions - implication or inferr :e, 
« LO, lower level questions - verba*^m or paraphrase, 

(c) The frequency of the adjunct questions asket^ (with two levels) 

« IN3ERT, questions inserted between text passages, 
« HASS, questiors massed at the end o£ ':he text. 

Dependent Variables 

There are two dependent variables 

(a) The level of the post-test item (with two levels) 

« HI, high level questions - inference and implication 
« LO, low level questions - verbatim and paraphrase 

(b) The relationship of the post-test question to the material 
questioned in the adjunct question. 

« INTENT, intentional material - related to material questioned 
by the adjunct question. 

« INCID, incidental material - related tc material not question 
by an adjunct question. 
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Thp Experimental Hypotheses 

The Main Effects Hypotheses 

These are the experimental hypotheses related to the main effects. 

(a) Presentation sequence and Intentional learning FREE « RQR > RRQ 

The freedom groups (FREE - representing real life self-study and distance 
education situations) and the read-question-read groups (RQR) are 
expected to use both backward and forward processing and better encode 
the information than are the read-read-qucstlon groups (RRGD which can 
only use backward processing. Thus the FREE and RQR groups are 
expected to recall more low level Intentional material and to perform 
better on high level questions about intentional material than the RRQ 
groups. 

(b) Presentation sequence and Incidental learning RRQ > RQR > FREE 

The RRQ groups read the passage twice and. because their attention during 
the second reading has not been focused by a question, are expected to 
recall more low order Incidental material and perform better on high level 
questions about Incidental material than are the RQR groups who are 
focused on the Intentional material during the second reading by a 
question. The RQR groups are expected to do better on questions about 
Incidental material than the FREE groups who may read the question first 
and, as indicated in the pre-question research, may merely forward process 
the intentional material ignoring the Incidental material. 

(c) Level of adjunct question and high level posttest Items HI > LO 

Inference and implication questions are expected to facilitate deeper 
semantic processing than verbatim and paraphrase questions. Thus the 
higher level adjunct question groups (HI) are expected to perform better on 
high level post test questions than the low level adjunct question groups 
(LO). 
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(d) Level of adjunct question and low ic^^el posttest items HI = LO 

High level adjunct questions are exr^ected to cause subjects to attend to 
more material, process it more deeply and have better organized memory 
than those asked low levr;l adjunct questions. Thus the high level groups 
(HI) should recall as much low level material as the low groups <L0) even 
though not specifically asked low level adjunct questions. 

(e) Frequency of adjunct question for all posttest items INSERT > MASS 

Because there is less chance of a match between text and question as the 
total amount of text between a :^articular target sentence and the related 
adjunct question increases, adjunct questions inserted after every 
paragraph are expected to facilitate better recall and performance than ali 
adjunct questions massed at the end. 



The Interaction Hypotheses 

There are no interaction hypotheses for two reasons. 

« There are no consistent research results to establish a theory, and 

« The more recent reviews (Hamilton. 1985; Hemaker, 1986) do not 
suggest strong or consistent interaction effects. 

Thus the factorial design is used to explore the possibility of interaction 
effects. 
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Method 
Subjects 

The subjects were 187 students (attrition from 195) from a large high school in 
Central New York. The school was selected because the principal and faculty were 
willing to cooperate. The subjects ranged from good grade 12 students to lower 
middle grade 10 students, and all were taught English by the two cooperating 
teachers. The researcher would have preferred to increase the power of the study 
by using all of the grade 12 students within the school because the variance would 
be lower, grade 12 students would be more cooperative, and the variance in an entire 
grade twelve would be similar to the variance in first year distance learners in 
higher education. The school was unable to provide the requested sample. 

All 195 students were required to participate and they were randomly assigned to 12 
treatment groups and one control group - about 15 per group. The members of a 
second control group were those absent from the treatment but were present for the 
posttest. This control group of 17 students was not randomly assigned and the 
researcher has not determined how representative the members were. 

The assignment procedure lised to place the 195 in the thirteen treatment groups 
involved using a pack of cards. The pack was shuffled and the cards dealt face up 
on a table. Each time a spade showed, a booklet from the treatment group assigned 
the number on the card was placed in the "assigned" pile. As there were 13 
treatments and 13 spades, each treatment wes assigned once per shuffle. The first 
booklet from each treatment was mixed in this way, then the second, then the next, 
and so on until all the booklets were randomly mixed. The random mix was taken to 
the classrooms and handed out to each class row by row. 

Design 

The experimental design was a posttest only control group design. The two 
statistical designs of the experiment can be represented as a 3 x 2 x 2 
between-subjects factorial design, and a one-way design with 12 groups and two 
separate control groi.'ps. For the factorial design, the between-subject factors 
were the presentation sequence (read-question-read, read-read-qucstion, and 
complete freedom), the level of the adjunct question (high and low), and the 
frequency of the adjunct questions asked (Inserted and massed). 
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The data from the experiment were analyzed using a 3 x 2 x 2 factorial analysis of 
variance and comparisons among the treatment groups were made using the Duncan 
post hoc test. A one-way analysis of variance with 14 levels was used to compare 
each treatment with the other treatments and with the two control groups and again 
the Duncan test was used. The Duncan test was chosen because the study was 
exploratory and the Duncan Is not as conservative as the Scheffe or Tukey tests. 

The dependent variables can also bt represented as a 2 x 2 factorial design. The 
factors in the design were the level of the posttest Item (high level and low level) 
and the relationship between the posttest Items to the material covered by the 
adjunct questions (Intentional and Incidental material). CThe Ideal statistical 
treatment for this design would have been MANOVA. However, this researcher Is 
unable to use MANOVA.] 

The Instructional Task and Materials 

The instructional material used In this study was an excerpt taken from a booklet 
entitled "The Energy Story" written by this researcher some years ago for use In high 
schools. The excerpt was broken Into 17 passages of between 100 and 130 words. 

The researcher/author Identified the most structurally Important sentence, phrase or 
idea In each passage and developed two completion or short answer questions for each 
of these sentences, phrases or Ideas. One question, defined as low level, required 
the reader to recall or Identify something and was expected to Induce verbatim or 
paraphrase processing in subjects In the low level question groups. The other 
question, defined as high level, required the reader to make an Inference or find an 
implication and was expected to induce high level semantic processing In subjects 
from the high level question groups. 

To illustrate, passage 10 from the "Energy Story" Is presented below with the target 
sentence underlined. 

We can see the light and feel the heat radiated from the sun. This energy is 
absorbed by the land, oceans and atmosphere and the earth heats up. The earth 
reflects 30JS of this solar energy directly back into space and it radiates another 
47X into space directly as heat. The remaining 23% is radiated into space as heat 
indirectly through evaporation, preclpltat « -nd the food chain cycle. Some of the 
sun's energy is used to evaporate the oc«. to form clouds. Some is used to heat 
the atmosphere to cause winds. Only a very s mall percentage (0.023^) of the sun's 
energy goes Into the oroceas of sugar form ation in plants. 
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The adjunct questions for this passage are: 

1. Low level. 

How much of the sun's energy goes Into the production of sugars in 
plants? 

2. High level. 

What is so important about the fact that only a very small percentage 
of the sun's energy helps produce sugars in plants? 

The Treatments 

Subjects in the treatment groups were provided with an adjunct question about each 
passage they read. The members of the LO groups received a question, defined as 
low level, which required the readers to recall or identify something and was 
expected to induce verbatim or paraphrase processing. The members of the HI 
groups received a question, defined as high level, which required the reader to make 
an inference or find an implication and was expected to induce high level semantic 
processing. Subjects in the treatment groups received adjunct questions in 
differing frequencies. Subjects in the MASS groups received all of the adjunct 
questions massed together at the end of all the passagesc The INSERT groups 
received adjunct questions inserted between passages. 

Subjects in the treatment groups also received differing presentation sequences. 
The RQR groups read the passage (or passages), received the question (or 
questions), then received the passage(s) again. The RRQ groups were provided the 
passage, then the passage again, and finally the adjunct question. Although the 
FREE group were able to proceed in any fashion, they were advised to read the 
passage, answer the question, re-read the passage, and attempt the question again. 

The no-questlon control group spent approximately the same time on each passage 
as the treatment groups but received no questions (the read twice control group). 
The no-passage control group received neither the questions nor the passages. 

The booklets for the 13 treatment groups are described below. 

Read-question-read, inserted-question groups (RQR-H-I and RQR-L-I). 

Each passage was presented twice with the high or low level adjunct 
question placed between them. The instructions informed subjects 
that they could not take notes and that once they had turned a page 
they could znot turn back. 
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Read-question-read, massed-questions groups (RQR-H-M and RQR-L-M). 

All 17 passages were presented once, then all the questions massed 
together, then the passages were presented again. The instructions 
were the same as the above. 

Read-read-question, inserted-question groups (RRQ-H-I and RRQ-L-I). 

Each passage was provided twice before the high or low level question 
was provided. The instructions informed subjects that they could not 
take notes and that once they had turned a page they could not turn 
back. 

Read-rcad-question, massed-questions groups (RRQ-H-M and RRQ-L-M). 

All 17 passages were presented once, they were presented again, then 
all the questions were presented massed together. The instructions 
were as above. 

Freedom, inserted-question groups (FR-H-I and FR-L-I). 

Each question was presented on the same page as the passage to which 
It refers. Subjects were allowed the freedom to proceed as they 
wished althouqh the following procedure was recommended: 

- Read the passage; 

- Read the question; 

- Re-read the passage; and 

- Answer the question. 

Freedom, massed question groups (FR-H-M and FR-L-M). 

All the passages were provided once, then all the questions were 
provided at the end of the booklet massed together. Subjects were 
allowed the freedom to proceed as they wished although the following 
procedure wa^ recommended: 

- Read the passages all together or singly; 

- Read the questions at the back of the booklet; 

- Re-read the passage(s); and 

- Answer the questions at the back of the booklet while lookirrg 
back at the qestlon. 

Read twice control group (RTCG) 

Each passage was provided twice. The Instructions Informed subjects 
that they could not take notes and that once they had turned a page 
Q they could not turn back. 
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No passage control group (NPCG) 

The subjects in the no passage control group were those students who 
were absent from class on treatment day but present on the posttest 
dtv". They received no treatment. 

Each adjunct question included a single line on which the answer could be written. 
This was included to help reinforce that long detailed answers were not required 
and to help persuade the students to answer the questions. 

Tests and Measures 

The oependent measure was made up of 34 multiple choice questions. 17 questions 
were based on the target sentence, phrase or idea in each passage - the intentional 
material. The other 17 questions were based on another sentence, phrase or idea in 
the passage - the incidental material. 17 of these questions, one for each of the 17 
passages, were high level questions requiring the subject to choose among 
Inferences or implications. The other 17 questions were low level questions 
requiring the subject to either choose the exact duplicate of the passage sentence or 
phrase or recognize a factual statement from the passage. 

A coin was tossed, once for each passage, to randomly determine s*'>ther the 
intentional material question was high level or low level. Each subject answered 9 
intentional high level questions, 8 intentional low level questions, 8 incidental high 
level questions and 9 iiicidental low level questions. 

As an illustration, the posttest questions for paragraph 10 above were: 

1. High level intentional question. 

Only 1 /4000th (0.023V) of the sun's energy that reaches earth goes into the 
process of sugar formation in plants. According to the passage, what is most 
important about chis fact? 

A. Plants do not need very much energy to make sugars and, thus, are very 
efficient organisms. 

B. Fossil fuels such as oil, coal and natural :as are formed from dead plants 
and animals (which eat the plants) and take thousands of years to form. 

C. We need to find an energy source that taps a higher percentage of the sun's 
energy and also makes that energy immediately available. 

D. Both B and C above. 
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2. Low level incidental question. 

Which of the following is a sentence from a passage you read? 

A. The sun reflects Al% cf this solar energy directiy back into space and it 
radiates another 30% into space directly as heat. 

B. The sun reflects 30% of this solar energy directly back into space and it 
radiates another 47% into space directly as heat. 

C. The sun reflects 30% of this solar energy directly back into space and it 
radiates another 23% Indirectly through evaporation and precipitation. 

D. The sun radiates 47% of this solar energy directly back into space and it 
radiates another 23% indirectly through evaporation and precipitation. 

Procedures. 

The experiment was conducted in the English classrooms of the two cooperating 
teachers on a Friday. The treatment booklets, already randomly mixed as described 
above, were already plared face down on the desks* when the subjects arrived for 
class. The subject? were told to read their special instructions, to follow only 
those instructions and to ignore what other students were doing as everyone had 
different instructions. As the subjects worked through the materials the 
cooperating teacher copied the treatment codes onto a class list. Subjects worked 
at their own rate and read their English texts if they finished before time. The 
treatments were completed within the 41 minute class period. 

The retention posttest was run 4 days later on the following Tuesday. It was a 
multiple-choice test as described above and the subjects marked a special answer 
sheet. As the subjects worked, the cooperating teacher transferred the treatment 
code from the class list to their answer sheets. If subjects finished the test 
before time, they were asked to check their answers and to write a comment about 
the experience on the back of the answer sheet. All subjects completed the test in 
the 41 minutes allowed. 

Students who wre absent for the treatment but present for the posttest still took 
the posttest. They were given the special code NPCG - the no passage, no question 
control group. As noted above, this control group was not randomly selected and 
its representativeness can not be assessed. 
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The Power of the Study 

As can be seen from Tables 1 through 4» the post test variance in each group is large, 
probably because the subjects ranged from poor grade 10 students to good grade 12 
students. The power of the study was greatly reduced by this large within-group 
variance, and few significant results were obtained. A complete range of grade 12 
students would have provided the variability needed for good external validity and 
would have Increased the power of the study by having less within-group variance 
than this sample from gradas 10, 11 and 12. 

To increase the possibility of significant results being detected, and to decrease 
the possibility of a Type II error, the Duncan post hoc test was used because it is 
less conservative than the Scheffe, Tukey or Newman-Keuls tests. To the same 
end, the F-test alpha value was set at O.i instead of 0.05 as is typically done. It 
should be noted, however, that when the F-test found significance between 0.10 and 
0.05, both the Duncan test and pair-wise comparisons of LS means found significance 
at the 0.05 level. 

The Analysis 

A 3x2x2 analysis of variance was performed on all nine dependent variables which 
were the 4 subscales of the post test and the various combinations of these 
subscales. The nine dependent variables, and the tables showing the results, are: 



(a) 


Intentional 


learning 




High level items 




(Tables 1 and ii>. 


(b) 


Intentional 


learning 




Low level items 




(Tables 2 and 12). 


(c) 


Incidental 


learning 




Hig^ level items 




(Tables 3 and 13). 


(d) 


Incidental 


learning 




Low level items 




(Tables 4 and 14). 


(e) 


Intentional 


Learning 




Total score [a ^ bl 


(Tables 5 and 15). 


(f) 


Incidental 


learning 




Total score Ic 4 


dl 


(Tables 6 and 16). 


<9) 


High level 


items 




Total score la 4 


cl 


(Tables 7 and 17). 


(h) 


Low level Items 




Total score lb 4 


dl 


(Tables 8 and 18). 


(i) 


Total score 


on all items [a b c -i* dl 


(Tables 9 and i9>. 



A one-way analysis of variance with 14 levels was performed to compare the 12 
treatments with each other and with the two control groups (see Table 10). Again 
the Duncan post hoc procedure was used (see Tables 11 through 19). 
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Presentation Sequence 

For high level items on intentional material, as hypothesized, the RQR groups 
to « 54.81) performed significantly better <p = 0.029) than the RRQ groups (x « 49.54) 
(see Tables 1 and 11). The prediction that the FREE groups (x « 52.15) would also 
score significantly higher than the RRQ group was not supported. It is important to 
note that the FREE group does not perform significantly worse than the RQR group. 

For posttest items requiring recall of low leve? material, the RQR groups (68.33), the 
RRQ groups (71.35) and the FREE groups (71.68) did not differ significantly. Also, 
for posttest items requiring high level perforTtance on incidental material, the RRQ 
groups (61.20), the RQR groups (60.83) and the FREE groups (58.67) did not differ 
significantly. Thus, for low level intentional items and high level incidental items 
all groups may be considered equal (see Tables 2, 3) and the null hypotheses accepted 
with little chance of a Type II error. 

For low level item3 on incidental material the hypothesis that the RRQ groups would 
recall more incidental material than the RQR groups, which would recall more than the 
FREE groups, was not supported. Rather, the FREE groups (43.31) recalled 
significantly more low level incidental material (p « 0.029) than did the RQR groups 
(37.53), with the RRQ groups (41.90) not differing significantly from the others (see 
Tables 4 and 14). The data support the opposite conclusion that FREE groups will 
recall more low level incidental material than will RQR groups. 

When the subscales were combined to form total scores on intentional and incidental 
material, there were no significant differences among groups for presentation 
sequence. Also, there were no significance differences when the subscales were 
combined to form a total score for high level items. 

When the subscales were combined to obtain a total low level score, significant 
differences were obtained (see tables 8 and 18). The FREE groups (x « 56.66) 
recalled significantly more low level material (p « 0.0273) than the RQR groups 
(X « 52.03) with the RRQ groups (x « 55,76) not differing significantly from the other 
groups. The prediction that the FREE groups would not recall as much as the other 
groups was not sup»...rted. In fact, the hypothesis was significantly contradicted: 
The data support the alternative hypothesis that FREE groups recall more low level 
material than RQR groups. 

No significant differences for presentation sequence were found when the subscales 
were combined to form a total posttest score. 

Er|c 30 
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Level of Adjunct Questions 

The first hypothesis related to the level of adjunct question was that, on high level 
posttest Items, the HI groups (which received high level adjunct questions) would 
perform better than the LO groups (which received low level adjunct questions). 
Although there were no significant differences, the means were ordered in the 
opposite direction. Hence, the experimental hypothesis Is not supported (see Tables 
1, 3, 7) and the null hypothesis, that there was no difference between the HI and LO 
groups, must be accepted with little chance of a Type II error. 

The second hypothesis related to the level of adjunct question was that, on low level 
posttest Items, the HI groups would score as well as the LO groups. As no 
significant dlffere.xes between the HI and LO groups were found on low level Items 
(see Tables 2, 4, 8), the experimental hypothesis Is supportsd and the statistical null 
hypothesis, that there are no differences between the HI and LO groups, must be 
accepted with little chance of a Type II error. 

Frequency of Adjunct Post-question 

The experimental hypothesis related to frequency of question was that adjunct 
questions are expected to facilitate better recall and performance If they are 
Inserted In the text than If they are massed together. This experimental hypothesis 
was not supported as no significant differences were found for any of the nine 
dependent variables - the difference between the INSERTed and MASSed groups was 
never greater than 2% (see Tables 1 through 9), and any small difference tended to 
favor the MASSed groups. Thus the null hypothesis, that there Is no difference 
between groups answering Inserted questions and groups answering massed 
questions. Is accepted with nttle chance of a Type II error. 

Interaction Effects 

There were five significant Interactions, for a significance level chosen at p < 0.10, 
all for high level posttest Items. The first three were QxF Interactions, that is. 
Interactions between level of adjunct question and whether those questions were 
Inserted In the text or massed together. One Interaction effect was found for high 
level Items about Intentional material (F » 2.69, p « 0.10), another for high level 
Incidental items (F « 3.78, p « 0.054) and a third for the total score on high level 
items (F « 5.35, p » 0.022). In all three cases, high level adjunct questions 
facilitated better performance on high level Items when massed, while low level 
adjunct questions facilitated better performance on high level Items when inserted. 
(See Tables 1, 3, 7, 11, 13, and 17^ 
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The other two significant interactions were found for the interaction between all 
main effects, PxQxF, that is, an interaction between presentation sequence, level of 
adjunct question and whether the adjunct questions are inserted in the text or 
massed together. One interaction was found for high level items about intentional 
material (F « 1.95, p « 0.14) and the other for the total score on high level posttest 
items (F « 2.30, p « 0.10). In both ca?es, high level adjunct questions facilitate 
better performance if massed and low level adjunct questions facilitate better 
performance if inserted, for RRQ and RQR groups but not for FREE groups. (See 
Tables 1,7, 11 and 17c) 

comparison with the Controls 

For high level posttest items about intentional material (Tables 10 and 11), although 
scoring between 53.17 and 48.48, three FREE groups, three RQR groups and two RRQ 
groups did NOT perform significantly better than the READ-TWICE control group 
(41.48) when the significance level of the Duncan post hoc test was left at 0.05. 
This researcher did not know how to alter the Duncan test to a 0.10 significance 
level and perhaps the 20*4 difference between the read-twice control and the lowest 
treatment group would have been significant at that level. All treatments 
performed better than the NO-PASSAGE control group (26.50). 

For low level intentional items (Tables 10 and 12), only the FREE-LO-MASSED 
group (73.86) recalled significantly more intentional material than the READ-TWICE 
control group (59.17). Because of the large within cell variance, the 
FREE -HI-MASSED group (72.92), the RRQ-LO-INSERTED group (72.73) and the 
FREE-LO-INSERTED group (71.43) were not considered significantly deferent from 
the READ-TWICE control group by the Duncan post hoc test set at a significant 
level of 0.05. Again these groups may have been significant at the 0.10 exploratory 
level. However, all treatment groups recalled significantl" more than the 
NO-PASSAGE control group (42.31). 

For high level incidental material (Tables 10 and 13), no treatment performed 
significantly better than the READ-TWICE control group. The RQR-LO-MASSED 
group did not perform significantly better than the NQ-PASSAGE control group. 



Inplications of the Adjunct Question Paradigm Page 30 

For low level incidental material (Tables 10 and 14), only the FREE-LO-MASSED 
vjroup <47.47) recalled significantly more than the READ-TWICE control group 
<34.07). Only two FREE groups and two RRQ groups recalled significantly more than 
the NO-PASSAGE control group (29.06). 

For all intentional items (Tables 10 and 15), three FREE groups, three RQR groups 
and two RRQ groups (all groups above 59.5) recalled significantly more and 
performed significantly better than the READ-TWICE control group (49.80). All 
treatment groups recalled significantly more and performed significantly better 
than the NO-PASSAGE control group (33.94). 

For all incidental items (Tables 10 and 16), while ALL treatment groups recalled 
significantly more and performed significantly better than the NO-PASSAGE control 
group (33.94), NO treatment groups (even RRQ-HI-MASSED at 53.78) recalled 
significantly more or performed significantly better than the READ-TWICE control 
group (43.14). 

For all high level Items (Tables 10 and 17), while all treatment groups performed 
significantly better than the NO-PASSAGE control group (32.58), only two RQR 
groups and one RRQ group (those above 58.00) performed significantly better than 
the READ-TWICE control group (47.06). 

For all low level Items (Tables 10 and 18), while all treatment groups recalled 
significantly more than the NO-PASSAGE control group (35.29), only th-ee FREE 
groups and two RRQ groups (those above 56.00) recalled significantly more than the 
READ-TWICE control group (45.88). 

For all Items, the total score (Tables 10 and 19), while all treatment groups 
performed significantly better than the NO-PASS AGE control group (33.94), only 
three FREE groups, two RRQ groups and two RQR groups (those above 54.5) 
performed better than the READ-TWICE control group (46.47). 
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INTENTIONAL LEARNING 



QF 





R Q R 1 




R R 


Q 1 


FREE 


df 1 




P 1 
1 


mean « 


54.81 1 


mean * 


49.54 1 


mean « 52.15 


1 2 1 


0.091 ! 


SD « 
n « 


13.48 1 
45 1 




SD « 

n » 


12.75 1 
48 1 


SD ' 10.44 
n = 49 


F 1 
1 2 . 44 


see 
table 
11 






HIGH 








LOW 


1 df 


P 


U 1 

1 


mean « 51.05 






mean = 53.27 


1 1 


0.261 


SD 
n 


« 12.86 
« 74 








SD « 11.76 
n « 68 


1 F 
1 1 .27 


ns 




INSERTED 








MASSED 


1 d 


P 


F 1 


mean » 51.54 






mean = 52 . 70 


1 1 


10.621 


SD 


" 11.99 








SD » 12.79 


1 F 


1 ns 


j 


n 


» 72 








n » 70 


10.25 




RQR-Hi 1 


RQR-Low 1 


RRQ'Hi 


RRQ-Low 1 


FREE-Hl 1 FREE-Lo 


1 df 


P 


PQ 1 


X » 52.78 1 


X » 57.14 1 


X 


* 50.43 


1 X « 48.48 


X > 50.00 1 X * 54.22 


1 1 


1 0.404 


8 - 12.37 1 


S - 14.61 1 


s 


' 14.48 


s « 10.59 


s * 11.81 1 s * 8.68 


1 F 


1 ns 




n « 24 1 


n - 21 1 




n * 26 


1 n • 22 


n > 24 1 n * 25 


10.91 




RQR-InsertI 


RQR-Nassedl 


RRO-Insert 


1 RRQ-Massed 


FREE-Insert IFREE-Mass 


1 df 


1 ^ 


PF 1 


X " 55.56 1 


X ' 54.04 1 


X 


« 46.86 


1 X s 52.00 


X « 52.14 1 7 » 52.17 


1 1 


10.410 


8 - 13.40 1 
n « 23 i 


S - 13.84 1 
n > 22 1 


s 


« 10.03 
n = 23 


1 s > 14.60 

1 n « 25 


t 8 « 11.23 1 s « 9.73 
1 n > 26 1 n > 23 


1 F 
10.90 


1 ns 



High 
X » 
SD • 
n * 



T 



• Inserted I High •> massed 
48.77 I X » 53.22 



11.04 
36 



For 



so 

n 



14.19 
38 



1 1 1 n I 

inR-B-IIRQR-H-HllM»-MI)nR-L-l!iI»Q-R-IIR{»-H-N 

I I I I I 
50.00 I 55.56 I 61.62 I 52.22 I 45.37 I 54.76 

I I I I I 
7.49 I 15.71 I 16.00 I 11.77 I 11.07 I 15.99 
I i I I I 
I n » 121 n « 121 0 « ill n « 101 n > 121 n - 14 
I I I I I I 



Low - Inserted 

X ' 54.32 
SD « 12.39 

n • 36 

T 



T 



Low - massed 
X • 52.08 
SD » 11.09 
n « 32 



df 
1 

F 

2.69 

1 



T 



Presentation Sequence 



RRQ-L-liRRQ-L-HiFR-B-I IFR-H-M IFR-L-I ire-L-H I df I p 

I I I I I I I 
48.48 I 48.48 I 50.93 I 49.07 j 53.17 j 55.56 j 1 j0146 1^ 

8.99 I 12.45 I 13.":3 I 10.00 I 8.91 I 8.61 I F I see I 
I I I ] I I liable I 
n • 111 n « Itin « 12 I n « 121 n « 141 n « 111 1.951 11 I 
I I I I I I I I 

Question Level F = Frequency of Question 
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TABLE 2: DEPENDENT VARIABLE 2. LOW LEVEL - INTENTIONAL LEARNING 



PQ 



PF 



QF 



T 



T 



R Q R 
mean « 68.33 
SD « 14. 5C 
n • 45 



1 R R 


Q 1 


1 mean « 


71.35 1 


1 SD " 


14.11 1 


1 n « 


48 1 



FREE 
mean « 71 . 68 
SD « 15.68 
n = 49 



I 



df 
2 
F 



I 

10.69 



HIGH 



mean 

SD 
n 



69.76 
14.63 
74 



LOW 
mean « 
SD « 
n ■ 



71.32 
14.97 
68 



df 

1 

F 



INSERTED 
mean « 70.66 
SD " 14.56 
n « 72 



1 1 

I RQR-Low I RRQ-Hi 

I X « 69.05 I X « 70.67 

' a « 15.62 I s > 13.67 

24 I n » 21 I n « 26 
I I 

I I 

1 I 



RQR-Hl 

X « 67.71 

8 « 13.75 I 
I 



10.35 

.1 



MASSED 
mean <b 70.36 
SD - 15.07 
n - 70 



I 



d 
1 
F 



10.01 



RRO-Low 
X » 72.16 
s « 14.91 
n s 22 



RQR-InsertI RQR-Nassedl RRQ-Insert 

X » 6y.57 I X = 67.05 I x » 72.28 ! 

I I I 

8 « 14.51 I a « 14.71 I a > 14.08 I 

I I I 

n-23 1 n-22 1 n>23l 

I I ! 

I I 



RRQ-Hassed 
X « 70.50 
a « 14.38 
n « 25 



r 

FREE-Hl I 
X « 70.83 I 
a » 
n 



FREE-Lo I df 
I 

( « 72.50 I 1 
I 

16.76 I 9 ' 14.88 I F 

I I 
- 24 I n - 25 10.61 
I I 

I I 

1 I 

FREE-InsertiFREE-Hass I df 

• _ I 

X » 70.19 I X - 73.37 I 1 

i I 

a « 15.44 I 9 « 16.12 I F 

I I 

n - 26 I n « 23 10.61 

I I 

I I 



High - ln9erted 
X » 70.14 
SO - 15.61 
n - 36 

T 



T 



High - massed 
x » 69.41 
SD - 13.85 
n - 38 



I 
I 

! ^ 



Low - Inserted 
X « 71.18 
SD « 13.63 
n ■ 36 



Low - massed 
X « 71 .48 
SD « rJ.57 
n - 32 



df 
1 



T 



T 



T 



I F 
I 

I 0.04 

I. 



1 — r 



iR»-H-llFQR-H-MiRQR-L-liRQR-L-MIRRQ-H-IIRRQ-H-«ll^^ jFR-H-M jFR-L-I jfR-L-M 

1 69.79 I 65.63 I 69.32 I 68.75 I 71.88 I 69.64 I 72.73 I 71.59 j 68.75 j 72.92 I 71.43 j 73.86 j 1 j0.914 

^ I 12.45 I 15.19 I 17.11 I 14.73 I 16.10 I 11.72 j 12.27 j 17.76 j 18.84 j 14.92 j 12.43 I 18.07 | ' | 

i n - 121 n • 121 n » 111 n « 101 n « 121 n > 141 n « 111 n « llln > 12 I n « 121 n « 141 n « 111 0.091 
I I I I I I I I • I I I I ' 

P ■ Presentation Sequence Q « Question Level F « Frequency of Question 
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TABLE 3: DEPENDENT VARIABLE 3. HIGH LEVEL - 
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INCIDENTAL LEARNING 



F 



PQ 



PF 



QF 



T 



R Q R 1 


R R 


Q 


1 FREE 


1 df 1 


P 1 


mean = 


60.83 1 


mean » 


61.20 


1 mean = 58.67 


1 2 


0.7321 


SD - 
n « 


17.80 1 
45 1 


SD » 

ti ^ 


18.46 
48 


1 SD s 14.92 
1 n <3 49 


1 F 
10.31 


ns 




HIGH 






LOW 


1 df 


P 


mean «= 59.63 






mean e 60.85 


1 1 


0.674 


SD 
n 


» 16.36 
» 74 






SD » 17.80 
n » 68 


1 F 
10. 18 


ns 


INSERTED 






MASSED 


1 d 


P 


mean » 60.24 




mean » 60.18 


1 1 


0.870 


SD 
n 


» 17.34 
» 72 






SD " 16.80 
n » 70 


1 F 
10.03 


1 ns 


ROR-Hi 1 


RQR-Low 1 


RRQ-Hi 


! RRQ-Low 


! FREE-Hi 1 FREE-Lo 


1 df 


P 


X « 59.90 1 


X > 61.90 1 


X = 60.58 


1 X « 61.93 


1 X * 58.33 1 X - 59.00 


] 1 


0.9811 


s « 15.63 1 
n • 24 i 


8 « 20.34 1 
n < 21 1 


8 « 16.84 
n ■ Zd 


1 8 s 20.59 
1 n « 22 


1 8 ■ 17.16 1 8 ■ 12.77 
1 n « 24 1 n >= 25 


1 F 


ns 1 


RQR-Insert 1 


RQR-Massedl 


RRQ-Insert 


1 RRQ-HdSsedlFREE-InsertlFREE*Hass 


1 df 


1 ^ 


X « 61.96 1 


X « 59.66 1 


X « 59.24 


1 X * 63.00 


1 X « 59.62 1 X « 57.61 


1 1 


10.600 


8 ■ 18.65 i 
n > 23 1 


8 « 17.22 1 
n « 22 1 


8 « 17.76 
n « 23 


1 8 » 19.26 
1 n « 25 


1 8 > 16.32 1 8 « 13.45 
1 n > 26 1 n « 23 


1 F 
10.51 


1 ns 



High - inserted 
X « 56.94 
SD - 15.65 
n - 36 



T 



High - massed 
X » 62.17 
SO > 16.82 
n - 38 

T 



T 



^1 

IRQR-B-I lR(fi-B-HIRQR-L-I IRQR-L-NIRRQ-B-I IRRO-B-M 
I I I I i I 
I 55.21 I 64.58 I 69.32 I 53.75 I 57.29 I 63.39 
For i I i I I i 

I 14.56 i 15.84 I 20.44 I 17.73 I 12.45 i 19.36 
I I I i i i 
I n > 121 n > 121 n ' 111 n « 101 n > 121 n « 14 
I I I I I I 



Low - inserted 

X » 63.54 
SO « 18.51 

n > 36 



T 



T 



Low - massed 
X » 57.81 
SD ■ 16.73 
n > 32 




P 

0.054 

see 

table 
13 



T 



df I p 



1 1 n 

RRO-L-IIRRQ-L-BIFR-B-I IFR-B-H IFR-L-I IFR-L-H ^ ^ 

61.43 I 62.50 I 58.33 I 58.33 I 53.17 I 60.71 I 1 10.259 

i I I I i I.I 
22.68 I 19.36 I 20.18 t 14.43 I 8.91 I 12.84 I F I 

I I i i I I I ns 

D > 111 D s lilD > 12 I D « 121 n s 141 n > 111 1.361 

I I I ! I I I 



Presentation Sequence 



Question Level 



F " Frequency of Question 



ERIC 



36 



TABLE 4t 
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DEPENDENT VARIABLE 4. LOW LEVEL - 
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INCIDENTAL LEARNING 





1 1 
1 R Q R 1 


R R 


Q 1 


FREE 


1 df 1 


P 1 


p 


1 mean « 


37.53 1 




41.90 1 


mean » 43.31 


1 2 1 


0.0841 


1 SD « 
1 n « 


14.27 i 
45 1 


SD » 
n » 


12.81 1 
48 1 


SD - 1 1 . 62 
n « 49 


1 F 1 
12.521 


see 1 

tab! e 1 
14 1 






HIGH 






LOW 


1 df 1 


P 


Q 


1 mean « 41 . 14 




mean « 40.85 


1 1 1 


0.9631 


1 SD 


« 13.68 






SD « 12.65 


1 F 1 


ns 1 




1 n 


» 74 






n » 68 


10.001 




1 INSERTED 






MASSED 


1 d 


p 


F 


1 mean = 39.51 




1 mean 42.54 


1 1 


0.168 


1 SD 


« 12.49 






SD - 13.51 


1 F 


ns 




1 n 


» 72 






n » 70 


1 1 .92 




1 RQR-Hl 1 


RQR-Low 1 


RRQ-Hi 


RRQ-Low 1 


FREE-HI 1 FREE-Lo 


1 df 


P j 


PO 


1 X « 36.11 1 


X » 39.15 1 


X ' 43.16 


X • 40.40 1 


X • 43.98 1 X • 42.67 


1 1 


0.5651 


1 9 « 15.10 1 


9 «: 13.43 1 


s * 13.45 


9 « 12.15 


9 « 10.61 1 S » 12.70 


1 F 


ns 1 




1 n « 24 1 


n > 21 1 


n » 26 


n - 22 


n ' 24 1 n <= 25 


10.57 




1 RQR-InsertI 


RQR-Massedl 


RRQ- Insert 


1 RRQ-Massed 


FREE- Insert 1 FREE-Mass 


1 df 


P 1 




1 X « 37.20 1 


X = 37.88 1 


X s 38.65 


1 X = 44.89 


X = 42.31 1 X » 44.44 


1 1 


0.561 1 


1 8 • 15.57 1 
1 n - 23 1 


9 ' 13.13 1 
n « 22 1 


s « 12.02 
n • 23 


1 9 = 13.02 
1 n > 25 


9 * 9.44 1 s = 13.81 
n > 26 1 n <= 23 


1 F 
10.58 


ns 1 




1 High - Inserted 1 Hls^ • 


- massed 


1 Low - Inserted 1 Low - massed 


1 df 


P 1 


QP 


1 X > 41.05 


1 X a 


41.23 


1 X » 37.96 1 X » 44.10 


1 1 


0.1621 


t SD = 13.23 


1 SD * 


13.89 


1 SD « 11.69 1 SD ' 13.08 


1 F 


1 ns 




1 n - 36 


1 n « 


38 


1 n - 36 


1 n « 32 


1 1.97 



T 



T 



^ 1 r 

IRQR-H-IIRQR-H-MIRQR-L-IIRQR-L-IIIRRO-H-IIRRQ-B-I! 
I I I I I I 
I 36.11 I 36.11 I 38.38 I 40.00 I 40.74 I 45.24 
FOP I I I I I I 

t 15.80 I 15.08 I 16.00 I 10.73 I 11.92 I 14.6 I 
I I I I I ^1 
I n - 121 n - 121 n - 111 n « 101 n -< 121 n « 14 
I I I I I I 



T 



df 



T 



1 1 I I n 

RRO-L-IIRRO-L-HIFR-H-I IPR-H-H IFR-L-I iFR-L-H I 

i I I I I I I 
36.36 I 44.44 I 46.30 I 41.67 I 38.89 I 47.47 I 1 10.508 

I I I I I II 
12.26 I 11.11 I 10.42 I 10.73 t 7.23 L16.S5 I P I 

I I I I 1 I _l ns 
n * 111 n « llln - 12 i n • 121 n > 141 n - 111 0.681 



P K Presentation Sequence 



Question Level 



F * Frequency of Question 



FRir 



37 
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TABLE Si DEPENDENT VARIABLE 5. TOTAL SCORE - 
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INTENTIONAL LEARNING 



F 



PQ 



PF 



QF 



T 



R Q P 1 


R R Q 


{ 
1 


FREE 


1 df 1 


P 1 


mean « 61 • 18 1 


mean » 59.80 


1 mean 




61.34 


1 2 1 


0.707 


SD » 11.03 1 


SD « 10.18 


1 SD 


K 


10.47 


1 F 1 
















ns 


n « 45 1 


n * 48 


1 n 




49 


1 n OCT 1 

1 0 .351 




HIGH 


1 


LOW 






1 df 


p 


mean « 59.86 


j 


mean « 


61. 


76 


1 1 


0.287 


SD « 10.89 




SD « 


10. 


04 


1 F 
















ns 


n « 74 




n » 


68 


1 1.14 




INSERTED 




MASSED 




i d 


P 


mean = 60.54 




mean » 


61. 


01 


1 1 


0.817 


SD « 10.61 




SD = 


10. 


46 


1 F 
















ns 


n « 72 




n = 


70 


10.05 





RQR-HI 
X » 59.80 
s « 9.91 
n - 24 



RQR- Insert 
X » 62.15 
8 • 11.61 
n • 23 



RQR- Low 
X « 62.75 
s s 12.25 
n - 21 



RQR-Hassed 
X " 60.16 
s « 10.57 
n « 22 



T 



RRQ-Hi 
X = 59.95 
s « 11.29 
n - 26 



.RQ- Insert 
X s 58.82 
s - 9.71 
n » 23 



High - inserted I High - massed 

X > 58.82 I X > 60.84 
I 

SD - 10.98 I SO > 10.88 
I 

n • 36 I n • 38 

1 1 r' 



. ~l 1 

IRQR-H-I IRQR-B-NIRQR-l-I IRQR-L-NIRRO-B-I IRRO-B-H 
I I I I I I 
I 59.31 I 60.29 I 65.24 I 60.00 I 57.84 I 61.76 
FQFI I I I I I 

I 9.20 i 10.97 I 13.53 I 10.67 I 11.72 I 11.00 
I I I I i I 
I n * 121 n > 121 n > 111 n ' 101 n - 121 n « 14 
I I I I I I 



RRQ-Low 
X « 59.63 
s « 8.95 
n > 22 



RRQ-Hassed 
X -= 60.71 
s = 10.70 
n = 25 



FREE-Hi i FREE-Lo 

X « 59.80 i X ' 62.82 
I 

S > 11.85 I 3 » 8.94 
I 

n • 24 I n « 25 



I 

FREE- Insert I FREE-Mass 
I . 

X > 60.63 I X - 62.15 
I 

s « 10.62 j 9 ' 10.47 
n « 26 I n B 23 



Low - inserted 

X s 62.25 
SD > 10.08 

n » 36 



T 



T 



Low - massed 
X « 61.21 
SO - 10.12 
n - 32 



df 
1 

F 

0.35 



df 
1 

F 

0.48 



df 
1 

F 

0.73 



df 



T 1 1 

RRQ-L-IIRRQ-L-MIPR-B-I IFR-B-M IFR-L-I IFR-L-M 

I I I I i I i 

59.89 I 59.36 I 59.31 I 60.29 I 61.76 I 64.17 I 1 10.663 

I I I I I I I 

7.36 I 10.67 I 12.66 I 11.53 I 8.86 I 9.28 I F I 

I I I I I I I ns 

n « 111 n « llln > 12 I n « 121 n > 141 n » 111 0.411 



P » Presentation Sequence Q "Question Level F = Frequency of Question 
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TABLE 6t DEPENDENT VARIABLE 6. TOTAL SCORE - INCIDENTAL LEARNING 



PO 



PF 



OF 



T 



RQR-Hl 
X » 47.30 
s « 10.90 
n « 24 



RQR- Insert 
X « 48.85 
a « 13.56 
n « 23 



RQR-Low 
X « 49.86 
s « 12.56 
n « 21 



RQR-Hassed 
X = 48.13 
s » 9.54 
n * 22 



RRQ-Hi 
X = 51.36 
s « 11.35 
n * 26 



RRQ-Insert 
X » 48.34 
s « 12.91 
n « 23 



High - inserted 
X » 48.53 
SD « 11.80 
n « 36 



High - massed 
X « 51.08 
SD « 10.15 
n « 38 

T 



T 



"n 1 r 

IRQR-B-I IRQR-H-NIRQR-L-I IROR-L-MIRRO-H-IIRRQ-H-M 
I I I I I I 
I 45.10 I 49.51 I 52.94 I 46.47 I 48.53 I 53.78 
POP I I i I I I 
I 1 12.12 I 9.54 I 14.41 I 9.78 i 10.97 I 11.50 
i I I I I I i 

I n « 121 n « 121 n > HI n « 101 n > 121 n > 14 
I 1 I I I I 



RRQ-Low 
X = 50.53 
s « 14.25 
n 22 



RRQ-Hassed 
X = 53.41 
s * 12.12 
n « 25 



FREE-Hi 
X = 50.74 
8 « 10.67 
n « 24 



FREE- Insert 
X - 50.45 
s > 10.01 
n « 26 



FREE-Lo 
X = 50.35 
a » 8.33 
n s 25 



FREE-Mass 
X » 50.64 
s = 9.00 
n » 23 



Low 
X ■ 

SO = 
n « 



- inserted 
50.00 
12.38 
36 



T 



T 



Low - massed 
X = 50.55 
SD = 10.97 
n « 32 





RQR 1 


R R 


u 






ux 


P J 


p 1 


mean « 48.50 1 


mean « 


50.98 


1 mean 


= 50.54 1 


2 


0.5571 


SD « 11.64 1 
n « 45 1 


SD « 
n « 


12.63 
48 


1 SD 
1 n 


9 . 45 

« 49 1 


F 

0.59 


ns 1 




HIGH 






LOW 




Hf 1 


P 


1 Q 1 


mean = 49.84 






mean » 


50.26 


1 


0.7871 


SD « 10.98 
n » 74 






SD « 
n « 


11 .65 
68 


F 

0.07 


ns 1 




INSERTED 






MASSED 


d 


P 


1 F 1 


mean « 49.26 






mean « 


50.84 


1 


10.475 


SD « 12.03 
n - 72 






SD « 
n « 


10.46 

70 


F 
0.51 


1 ns 1 



df 
1 

F 

0.24 



df 
1 

F 
0.93 



df 
1 

F 

0.28 



P 

0.789 
ns 



P 

0.397 
ns 



P 

0.594 
ns 



1 I I I 

RRQ-L-IIRRQ-L-HIFR-H-I IPR-H-M IFR-L-I IPR-L-M I df I p 

I I I I I I I 
54.SC I 52.94 I 51.96 I 49.51 I 49.16 I 51.87 I 1 10.233 

I I I I I 11 
13.94 I 13.41 I 12.23 I 9.20 I 7.86 I 9.04 I F 

I I I I I I i ns 
n * HI n > Uln * 12 I n « 121 n > 141 n > HI 1.471 

I I I I I I I 



P « Presentation Sequence 



Question Level 



Frequency of Question 
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TABLE 7: DEPENDENT VARIABLE 7. HIGH LEVEL ITEMS - TOTAL SCORE 



F 



PQ 



PF 



OF 



R Q R 1 


R R 


Q ! 


FREE 


1 df 1 


P 1 


&.ean « 


57.65 1 


mean « 


55.02 1 


mean » 55.22 


1 2 i 


0.4491 


SD = 
n » 


13.07 1 
45 i 

1 


SD = 
n = 


11.92 1 
48 1 


SD « 9.44 
n = 49 


1 F 
10.80 


■ 19 1 




HIGH 






LOW 


1 df 


P 1 


mean = 55.09 




mean >= 56.83 


1 1 


0.3491 


SD 
n 


« 10.33 
- 74 






SD « 12.68 
n « 68 


1 F 
10.88 




INSERTED 






MASSED 


1 d 


p j 


mean = 55.64 




mean = 56.22 


1 1 


0.8701 


SD 
n 


« 11.84 

= 72 






SD = 11.23 
n « 70 


1 F 


ns 1 


RQR-Hi 1 


RQR-Low 1 


RRQ-Hi 


RRQ-Low 1 


FREE-Hi 1 FREE-Lo 


1 df 


P 1 


X * 56.13 1 


X « 59.38 1 


X « 55.20 


X = 54.81 1 


X « 53.92 1 X « 56.47 


i 1 


0.7811 


8 « 9.96 1 
n « 24 1 


s = 15.99 1 
n * 21 1 


9 « 10.53 
n « 26 


s = 13.63 1 
n » 22 1 


s = 10.79 1 s e 7.96 
n > 24 1 n » 25 


1 F 
10.25 




RQR-Ini5ertl 


ROn-Massedl 


RRQ-Insert 


RRQ-Hassed 


FREE- Insert IFREE-Mass 


1 df 


P j 


X « 58.57 1 


X « 56.68 1 


X » 52.69 


X « 57.18 


X « 55.66 1 r. » 54.73 


1 1 


0.340 1 


s « 14.46 1 
n « 23 1 


s « 11.70 1 
n * 22 1 


s - 10.41 
n 3 23 


s = 12.99 
n « 25 


s « 10.15 1 8 « 8.77 
n = 26 1 n > 23 


1 F 
1 1.09 


ns 1 


High - inserted 1 High • 


- massed 


Low - inserted 1 Low - massed 


1 df 


P 1 


X « 52.61 


1 X « 


57.43 


X » 58.66 1 X « 54.78 


1 1 


0.0221 


SD « 8.78 


1 SD - 


11.23 


SD * 13.74 1 SD s 11.22 


1 F 


see 1 

Table 1 
17 1 


n ■ 36 


1 n « 


38 


1 n « 36 


1 n > 32 


1 5.35 



T 



^ I r 

IRQR-B-I IKQR-B-NIRQR-L-I IRQR-L-NIRKQ-H-I IRRO-B-N 
I i I I i I 
I 52.45 I 59.80 I 65.24 I 52.94 I 50.98 I 58.82 

POP I i I I I i 

I 7.79 I 10.89 I 17.34 I 12.09 I 5.79 I 12.42 

I i I i i I 

I n « 121 n ' 121 n « 111 n > 101 n > 121 n > 14 

I I I I I I 



1 1 1 1 I 1 I 

RRQ-L-IIRRQ-L-MIPR-B-I IFR-H-M IFR-L-I IFR-L-M I df I p 

I I I I I I I 

54.55 I 55.08 I 54.41 I 53.43 I 56.72 I 56.15 I 1 10.104 

i I I I i I I 

13.94 I 13.99 I 12.06 I 9.86 I 8.51 I 7.61 I F I see 
I I I I I I liable 

n » 111 n > llln = 12 I n > 121 n = 141 n > HI 2.301 17 

I I _l I I I I 



Presentation Sequence 



Question Level 



F = Frequency of Question 



A n 
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TABLE 


8: 






DEPENDENT 


VARIABLE 


8. 




LOW LEVEL ITEMS 


- 


TOTAL 


SCORE 








RQR 1 


R R 


Q 




1 

1 FREE 




1 at 1 


P 1 


i p 




mean « 


52.03 1 


lllCOll 


5S 76 

WW . r w 








1 2 


0.0721 






SD - 


9.04 1 


SD s 


A U . W V 




1 SD K 10.58 




1 F 


see 1 
Tablet 
18 1 








n » 


45 1 


n 






1 n K 49 




12.68 


} 








HIGH 








LOW 




1 dr 


P j 


! Q 






mean » 54.61 








m^An » 55-19 




1 1 


0.7031 






SD 


« 10.42 








SD « 9.91 




1 F 


1 ns 








n 


» 74 








n = 68 




iO.15 


1 






INSERTED 








MASSEL 




1 d 


P 








mean » 54. 17 






mean « 5b 63 




1 1 


1 0 . 390 






SD 
n 


» 10.07 
« 72 








SD « 10.25 
n = 70 




1 F 
10.74 


1 ns 


j 


1 RQR-Hi 1 


RQR-Low 1 


RRQ-Hi 


! RRQ-Low 


1 FREE-Hi 1 FREE-Lo 


1 df 


1 ^ 1 


( DO 

1 ru 


1 X 


m 


50.98 1 


X " 53.22 1 


X « 56.11 


1 X « 55.35 


1 X = 56.62 1 X » 56.71 


1 1 


1 0.7821 


1 s 


m 


10.36 1 


s » 7.32 1 


S « 9.59 


1 S » 1?.29 


1 s « 10.81 1 s « 10.59 


1 F 


1 ns 1 


i 




n 


« 24 1 


n » 21 1 


n « 26 


1 n » 


22 


1 n » 24 1 n » 


25 


iO.25 




1 RQR- 


-Insert 1 


RQR-Hassedt 


KKQ* insert 


1 RRQ-Massedl FREE- Insert 1 FREE-Mass 


1 df 

1 Ul 


D 1 


1 PF 


1 X 


m 


52.43 1 


X « 51.60 1 


X « 54.48 


1 X « 56.94 


1 X « 55.43 1 X s 58.06 


1 1 


10.679 1 


1 8 


m 


10.33 1 


s » 7.69 1 


s s 10.22 


1 s « 10.43 


1 8 « 9.87 1 S » 11.40 


1 F 


1 ns 1 






n 


« 23 1 


r « 22 1 


n « 23 


1 n » 


25 


1 n ' 26 1 n « 


23 


10.39 



OF 



T 



T 



High - inserted I High - massed 

X » 54.74 I X » 54.49 
I 

SD « 11.49 I SD » 9.44 

I 

n « 36 I n > 38 



T 



~T 1 r - 

IRQR-H-I IRQR-H-MIRQR-L-I IRQR-L-MIRRQ-a-I IRRQ-H-H 
I I i f I I 
I 51.o^> I 50.00 I 52.94 I 53.53 I S5.39 i 56.72 
POP I I I I i I 

iir.iOl 8.13 1 7.89 1 7.04 110.18 1 9.40 
I I I I I I 
I li » 121 n » 121 n > 111 n « 101 n > 12! n • H 
I I I I I I 



Low - inserted 

X » 53.59 
SD « 8.53 

n « 36 



Low - massed 
X = 56.99 
SD - 11.12 
n « 32 



T 



df 
1 

F 

1.16 



r>L-llRRO'L-l!iFR-H-I 'irS-H-M IFR-L-I IFR-L-M j | P 

53.48 I C\22 I 56.86 I 56.37 I 54.20 I 59.89 I 1 10.8775 

I I I I I ! . I 

lO.C? I 12.08 I 12.12 I 9.86 I 7.71 I 13.11 j P I 

I I I I I I i ns 

n « Hi n > llln > 12 I .i * 121 n • 141 n > 111 0.131 



J 



Presentat i>>n Sequence 



Question Level 



F = Frequency of Question 



ERIC 
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TABLE 9: DEPENDENT VARIABLE 9. ALL ITEMS - TOTAL SCORE 





1 R Q S 


1 

! 1 R R 


Q i FREE 


1 1 
1 df 1 


P 1 


p 


1 mean « 


54.84 1 cr-an * 


55.39 1 mean = 55.94 


t r% 1 
1 1 


0.8141 


1 SD - 


9.07 1 SD « 


9.31 1 SD = 7.68 


i IT 1 


ns 1 




1 n « 


45 1 n - 


48 1 n « 49 


10.21 1 






HIGH 1 


LOW 


1 df 1 


P 1 


0 


1 mean 54.84 1 


mean « 56.01 


1 1 1 


0.4071 


1 SD 


8.14 1 


SD " 9.20 


1 IT 1 
1 If 1 


ns 1 




1 n 


« 74 1 


n * 68 


10.691 




1 INSERTED 1 


MASSED 


1 d 1 


P 1 


F 


1 mean « 54.90 1 


mean « 55.92 


1 1 1 


0.5441 


1 SD 
1 n 


9.18 1 
« 72 1 


SD ^ 8.10 
n = 70 


1 F 1 
10.371 


ns 1 




1 i 
1 RQR-Hi 1 


i 1 
RQR-Low 1 RRQ-Hi 1 


1 1 
RRQ-Low 1 FREE-Hi 1 FREE-Lo 


1 df 1 


P 1 


PQ 


1 - 1 
: X « 53.55 1 


1 _ 1 
X = 56.30 1 X « 55.66 1 


X s 55.08 1 X « 55.27 1 x » 56.59 

A ™ WW . WW 1 ^ WW . f j WW • V » 


1 1 1 


0.7061 


1 1 
1 s > 7.80 1 


1 1 
8 ' 10.33 1 S « 8.19 1 


8 « 10.67 1 s « 8.58 1 S « 6.81 


1 F 1 


ns 1 




i 1 
1 n > 24 1 
1 1 
1 1 


1 1 
n « 21 1 n « 26 1 

1 1 
1 1 


n«22l n»24l n«25 
1 1 


1 ft OC 1 

1 0 . 99 1 




1 1 
1 RQR-InsertI 


1 1 
RQR-Massedl RRQ-InsertI 


1 1 
PRQ-MassedlFREE-Insert FREE-Hass 


1 df 1 


P 1 


PF 


1 X « 55.50 1 


X « 54.14 1 X « 53.58 
1 

S « 6.89 1 8 > 8.87 


X « 57.06 1 X « 55.54 1 x « 56.39 


1 1 1 


0.402 1 


1 1 
1 8 > 10.87 1 


1 1 

S « 9.57 1 S 7.99 1 S « 7.45 
1 1 


1 F 1 






1 1 
1 n - 23 1 
1 t 
1 1 


1 ! 
n s 22 1 n « 23 
1 
1 


n ' 25 1 n « 26 1 n ' 23 

1 1 


10.92 




1 i 

1 High - inserted 1 Mgh - massed 


Low - inserted 1 Low - massed 


1 df 


p 


QF 


1 X > 53.68 


1 X « 55.96 
( 

1 SD « 7.99 


1 X « 56,13 ! X « 55.88 


1 1 


0.3861 


1 SO « 8.24 


1 SD « 10.00 1 SO > 8.35 


1 F 


ns 1 




1 n - 36 


1 

1 n « 38 
1 


1 n » 36 1 n « 32 


1 0.76 


I I 1 1 1 1 
iSQR-H-I IKQR-B-NI RQR-L-I IIX2R-L-MIRR0-B-I IRRO-B-N 


Irro-l-iIrrq-l-mifr-b-i Ifr-b-m Ifr-l-i IFR-L-M ! df 1 p 

1 1 1 1 1 1 1 1 


1 

por 1 
1 

1 1 
1 

1 1 


S2.21 1 54.90 1 59.09 1 53.24 1 53.19 1 57.77 
1 1 1 1 1 

8.88 1 6.68 1 12.09 1 7.40 t 6.57 1 9.05 
1 i 1 1 1 

n > 121 n « 121 n « 111 n « 101 n « 121 n < k 
1 1 1 1 1 


1 54.01 1 56.15 1 55.64 1 54.90 1 55.46 1 58.02 I 1 

1 1 1 1 1 1 1 


10.267 


t 11.18 1 10.56 1 9.34 1 8.16 ( 7.00 1 6.59 F 

1 1 1 1 1 1 1 ns 
J > 111 n = Uln « 12 1 n » 121 n » 141 n < 111 i.33l 
1 1 1 1 1 1 1 1 



P » Presentation Sequence Q » Question Level F » Frequency of Question 



TMU 10: ieans for ALL Groups on MX Dependent Variables - t »alue for one «y AUOVA with 14 levels. 
Dependent Group 



Variable 


QRHI 


QRHH QRLI 




SQHI 


RQnM 


RQLI 


KQLH FKHI 


nun 
fKnn 


VDT t 
tKui 


nrii 


PTC6 

Kivv 


luvv 


Intent. 


50.00 


55*56 61 162 


52.22 


45.37 


54.76 


48.48 


48.48 50.93 


49.07 


53.17 


55.56 


41.48 


26.50 




df - 


13, 169 




F ■ 


5.70 




p < 0.0001 












Int«nt. 


69.79 


65.63 69.32 


68.75 


71.88 


69.64 


72.73 


71.59 68.75 


72.92 


71.43 


73.86 


59.17 


42.31 


low 


df > 


13, 169 




r ■ 


4.00 




p < 0.0001 












Ineld. 


55.21 


64.58 69.32 


53.75 


57.29 


63.69 


61.36 


62.50 58.33 


58.33 


60.71 


56.82 


53.33 


39.42 


Bl«f) 


df - 


13, 169 




r ■ 


2.10 




p - 0.0168 


< 0.05 










Incld. 


36.11 


36.11 38.38 


40.00 


40.74 


45.24 


36*96 


44.44 46.30 


41.67 


38.89 


47.47 


34.07 


29.06 


low 


df > 


13, 169 




r ■ 


1.92 




p - 0.0317 


< 0.05 










Intent. 


59.31 


60.29 65.24 


60.00 


57.84 


61.76 


59.89 


59.36 59.31 


0O.29 


61.76 


64.17 


49.80 


33.94 


Total 


df - 


13, 169 




P ■ 


6.81 




p < 0.0001 












Incld. 


45.10 


49.51 52.94 


46.47 


48.53 


53.78 


48.13 


52.94 51.96 


49.51 


49.16 


51.87 


43.14 


33.94 


Total 


u% ■ 


13, 169 




f ■ 


2.62 




p > 0.0026 


<0.01 










Total 


52.45 


59.80 65.24 


52.94 


50.98 


58.82 


54.55 


55.08 54.41 


53.43 


56.72 


56.15 


47.06 


32.58 


Hl(t> 


df ' 


13, 169 




r ■ 


5.26 




p < 0.0001 












Total 
low 


51.96 


50.00 52.94 


53.53 


55.39 


56.7? 


53.48 


57.22 56.86 


56.37 


54.20 


59.89 


45.88 


35.29 


df • 


13, 169 




r ■ 


4.53 




p < 0.0001 












Total 


52.21 


54.90 59.09 


53.24 


53.19 


57.77 


54.01 


56.15 55.64 


54.90 


55.46 


58.02 


46.17 


33.94 


Recall 


df " 13, 169 




r ■ 


6.59 




p < 0.0001 
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TABLE ilt SJtSm of SISilPICANT S^TS: VAS. 1 HfiafTIONAL MATB^IAL - BI6H LEVEL OUESTIONS 





r- test 
P 


Duncan post hoc LS Heans - Probability XI * X2 
p < O.05 p < 0.05 


p 1 


0.091 


1 1 
RQS > RRQ 1 SGR (54.81) > RRQ (49.54) p * 0.029 


QxP 1 


0.103 


fligt) is better Mssed - Low is better inserted 


POP 1 


0.146 


Bigl) is better ussed - Low is better inserted - for RTO and HOR but not FREE 








ALL 1 


0.0001 ! vay 


All treatments > VKG (control) 

Only RQR-L-I. FREE-L-N. m-M, and RRQ-B-H > KKG (contrul) 
RQR-L-I > RRO-H-I, RRO-L-I, RRO-L-M, FR-H-M, and RQR-H-I 



TABLE 12: SUKHARY of SluiIFI(»NT RESULTS: VAR. 2 IKTEffl'IONAL HATERIAL - UX LEVEL QUESTIONS 



P - test 
P 



Duncan post hoc 
p < 0.05 



LS Kean^ 



Probability XI « ){2 
p < 0 05 



1 

ALL I 0.0001 1 way 

I 



Only FR-L-M > RTCG 
All treatHF ' > HPCG 



TABLE 13t SUNNARy OF SICNIPICViT m,J.St VAR. 3 INCIDENTAL MATERIAL - HICH LEVEL QUESTIONS 



P - test 
P 



Duncan post hoc 
p < 0.05 



LS Means - Probability XI « X2 
p < 0.05 



QxP I 0.054 

I 



I 

AU I 0.U168 1 way 

» I 



High is better Mssed - Low is better inserted 



No treatments > KKG 

Al I treatments > NPCG except R(K-L-M 



TABLE 14: SUMMARY of SIGNIFICANT RESULTS: VAR. 4 INCIDENTAL MATERIAL - LOV LEVEL QUESTIONS 



P - test 
P 



Duncan post hoc 
p < 0.05 



LS Means - Probability XI « X2 
p < 0.05 



T 



P i 0.064 I FREE > RCR I FREE (43.31) > RQR (37.53) p * 0.0290 

J I I 



ALL 

erJc 



0.0317 1 way 



Only FR-L-N > RTC6 

Only FR-L-N, FR-H-I, RRO-H-M, and RRO-L-M > NPCG 
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UBU 15: SmXi of SIGNIFICMT RESULTS: VAR. 5 IHTENTIONAL MATERIAL - TOTAL SCORE 



r - test 
P 



Duncan post hoc 
p < 0.05 



LSKeans - Probability Xi > X2 
p < 0.05 



1 

ALL I O.OOOi i way 

I 

I 



Ail treatnents > RTCG except RRO-L-M, RQR-H-I, FR-H-I and RRQ-H-I 
All treatnents > NPC6 



TABLE 16: SUHNARV of SIGNIFICAHT RESULTS: VAR. 6 INCIDENTAL MATERIAL - TOTAL SCORE 



P - test Duncan post hoc 
p p < 0.05 



LS Means - Probability XI > X2 
p < 0/*5 



1 

ALL I 0.0026 1 way 

I 

I 



No treatments > RTCG 
All treatnents > NPCG 



TABLE 17: SUKMART of SIGNIFICANT RESULTS: VAR. 7 BIGB LEVEL ITEMS TOTAL SCORE 



P - test 
P 



Duncan post hoc 
p < 0.05 



LS Means - Probability Xi > X2 
p < 0.05 



QxP 
POP 



ALL 



0.022 



0.104 3x2x2 



0.0001 i way 



Higt) is better sassed - low is better inserted 



High is better aassed - Low is better inserted - for RRO and RQR but not FREE 



All treatments > f'lKG 

Only RQR-L-I, RQR-L-M, RRQ-H-M, > RTCG 

RQR-L-I > RR0-8-I, RQR-H-I, RQR-L-M AND FR-H-M 



TABLE 18: SUMMARY of SIGNIFICANT RESULTS: VAR. 8 LOV LEVEL ITEMS 



TOTAL SCORE 



P-test Duncan post hoc LS Means - Probability Xi > X2 

p p < 0.05 p < 0.05 



p 1 


0.072 1 


FREE > RQR i FREE (56.66) > RQR (52.03) 
1 


p B 0.0273 










ALL 1 


0.0001 1 way i 


Only FR-L-M, RRO-L-M, FR-H-I, RRO-H-M, and FR-H-M 
All treatments > NPCG 


> RTCG 
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nUE 19: SUMMARY of SIGNIFICANT RESULTS: VAR. 9 TOTAL SCORE ALL ITEMS 



r - test 
P 



Duncan post hoc 
p < 0.05 



LS Means - Probability XI > X2 
p < 0.05 



1 

ALL I 0.0001 1 way 

I 

I 



Only WR-L-I, PR-L-M, RRQ-H-H, RRQ-L-M, FR-H-I, FR-L-I, RQR-H-M > RTC6 
All treataents > NPC6 
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I2iasU9lifiQ 

Weaknesses in the Study 

The main weakness in the study is that it lacks power due to too great a 
wi thin-group variance. The cooperating principal, due to a misunderstanding, 
provided a very diverse sample which included the full range of poor grade 10 
through very good grade 12. The complete grade 12 would have provided a sample 
with enough variance to establish generallzability, there was no need to sample 
across the three grade levels. To identify possible differences, and to lower the 
chance of making & Type II error, the alpha level was set at ten percent - p = 0.10. 

Contributinr to this weakness is the fact that the Duncan significance level was pot 
altered from p « 0.05 (due to ignorance of this researcher) and thus differences at 
the p < 0.10 level were identified when the Duncan was used. This may explain why 
very few significant differences between the treatment groups and the 
READ-TWICE control group were found even when the differences between the 
means was over ten percentage points. 

The second weakness lay with the statistical analysis which was a 3x2x2 analysis of 
variance for each of nine dependent variables. The four main dependent variables, 
the four subscales of the post-test, were treated as if they were completely 
independent when they could be considered as being related factorlally, as a 2x2 
relationship, with level of post-test item and whether the item was related to 
intentional or Incidental material being the two main dependent variables. For this 
reason, and because nine dependent variables were derived from the four subscales, 
multivariate analysis of variance (MANOVA), and the Wilkes Lambda test, should 
have been employed. Again, the reason for this oversight was ignorance on the part 
of this researcher. Borg and Gall (1983, p.557> indicate that the significance of 
differences may be increased when interrelationships are taken into account with 
HANOVA. 

Further, in an attempt to study three independent variables using a factorial 
design, this researcher caused increased divergence in the main effects groups. In 
any replication this researcher will concentrate on only two of the independent 
variables - presentatior. aequence and level of adjunct question. 

Because the weaknesses in the study lowered its power, any differences that are 
found may be considered more credible. 
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Presentation Sequence and High Level Intentional Material 

It was hypothesized that, for high level Intentional learning, the FREE groups and tne 
RQR groups would perform better than the RRQ groups. The rationale, after Andre 
and Sola (1975) and Friedman and Rlckards (1981), was that when a reader first reads 
the passage it is encoded at a shallow level. Thus, the high level inserted post 
question is too difficult for the reader to answer and the desired higher level semantic 
processing does not occur. ^ 

Reading the passage twice, as in the. RRQ case, should raise the level of the initial 
encoding and make anrwering the highi^r level post question a little easier by improving 
the backward processing. However, Andre and Scia (1976) argue that re-reading the 
passage, after an initial attempt to encode it while attempting the question, is a 
better solution to the higher level encoding problem. The post question should 
enhance the forward processing of the passage during the second reading. Thus the 
RQR procedure should lead to better semantic encoding than RRQ, and is preferable to 
a prequestion because the learning of incidental material should still occur. 

The FREE group was advised to read the passage, read the question, re-read th5 
passage and answer the question. It was expected that this procedure, if followed, 
should be as effective as the RQR sequence because both backward and forward 
processing should be facilitated. 

As expected, the RQR groups did perform significantly better than the RRQ groups on 
high level intentional material. Because of the weaknesses in the study, the profound 
differences found by Andre and Sola were not replicated but the difference was 
significant. The Andre and Sola study was done with unconnected sentences while this 
study was done with connected discourse which generalizes better to real life and 
distance education. 

The expected difference between the FREE groups and the RRQ groups was not found. 
However, the FREE groups did not perform significantly worse than the RQR groups so 
the assumption that the FREE procedure is as effective as the kCiR procedure is 
supported for high level intentional material. The present procedures used by those 
distance educators and developers of self -instructional materials, who employ 
questions to help their learners process difficult prose material, can be continued with 
confidence. In fact, it is recommended that all distance educators, developers of 
self-study Qojurses and lectyrerr who suggest readings add these procedures to their 
repertoires when their students are required to learn from connected discourse in 
texts and other prose materials. 



Iwllcatlons of th« Adjunct Question Paradlgn 



Page 46 



Presentation Sequence and Low Level Intentional Material 

Although the RQR sequence was developed to help readers semantically encode 
higher level intentional material, it was expected that the RQR and FREE groups 
would recall more lower level material than the RRQ groups due to the specific 
forward processing induced by the inserted question. This was expected because 
those asked low order questions would focus on the intentional learning during the 
second reading while those asked higher level questions would encode low order 
material while functioning at a higher level. 

The prediction that RQR and FREE groups would recall more low level intentional 
material than RRQ groups was not supported. Friedman and Rickards (1981) found 
that their read-twice control groups were much superior to their read-once control 
groups, and this leads one to the conclusion that the RRQ groups may have done as 
well as the RQR and FREE groups because they read the material twice and encoded 
it at a level higher than the read-once groups. The RRQ sequence leads to specific 
backward processing of reasonably well encoded low level material and recall is 
facilitated. 

Presentation Sequence and High Level Incidental Material 

It was hypothesized that the RRQ groups would perform better on high level 
incidental items than the FREE and RQR groups. This was because the question 
caused specific backward processing of intentional material only after the reader 
had read the passage twice and had processed incidental material. This double 
reading should lead to better performance on high level incidental material whether 
the adjunct question was high or low level. Also the question causes the RRQ 
reader to approach the next double reading with general forward processing. They 
do not know what the question will be so they process all of the material carefully 
looking for patterns and information wnich may help them answer the unknown 
question. 

The RQR groups may not bother to attempt backward processing when they see the 
question after reading the passage once because they know that they are going to 
read the passage a second time. They are, therefore, less likely to recall incidental 
material or perform well on higher level items related to incidental material. Also, 
having read the question, they approach the second reading with specific rather than 
general forward processing and may again ignore incidental material. 
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The FREE groups, although they are given directions to read the passage, read the 
question, re-read the passage and answer the question, may actually merely read 
the question and hunt for the answer. Thus the FREE groups were expected to 
recall less Incidental material and to perform worse on high level items related to 
Incidental material than both the RRQ and RQR groups. 

For high level Incidental material, the expected RRQ > RQR > FREE order was 
achieved. However, the differences were not significant (only A% difference 
between RRQ and FREE with RQR in between) and the expected order probably 
occurred by chance. That the expected difference between the RRQ and RQR groups 
did not arise. Is difficult to explain. Perhaps the material was not difficult enough, 
or perhaps the connectedness of the discourse provided semantic cues which enabled 
the RQR (and FREE) groups to respond appropriately to the higher level incidental 
Items. 

In terr:s u£ the theoretical position stated above, that the expected difference 
between the RRQ and FREE groups did not arise Is even more difficult to explain. 
Perhaps the connectedness of the discourse helped as suggested above. Perhaps 
the FREE groups followed their Instructions exactly, read the passage carefully 
once, used backward processing to answer the question, and used forward processing 
du»*lng the second reading. It Is also possible that, in their attempts to answer the 
difficult inference and Implication adjunct questions, the FREE groups Iiad to read 
and re-read the material (possibly more than twice) In an attempt to encode the 
complete passage of connected discourse rather than concentrate on only the 
questioned material. 

Whatever the reason for the FREE group performing as well on high order incidental 
questions as the RRQ and RQR groups, distance educators, developers of 
self-Instructional materials, and faculty who provide study guides for their 
required texts can breath a little easier. They have evidence that, as long as they 
ask high level questions, allowing their students the freedom to lookback at the 
passages while answering the questions will not lead to the '*death of learning 
effects** suggested by Shavelson (i974b), Duchastel (1983) and Duchastel and 
Nungester (1984). 
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Presentation Sequence find Low Level Incidental Material 

For low level incidental material the expected RRQ > RQR > FREE order was not 
found. The order found was FREE > RRQ > RQR with only the FREE > RQR being 
significant. That the FREE group recalled the most low level incidental material was 
completer* unexpected and is all the more surprising due to low power of this study. 

rtothkopf (1965), in developing the adjunct question paradigm, argued that it is the 
Incidental learning that is more educationally significant and, on the assumption that 
allowing readers the freedom to read as they wish would lead to only intentional 
material being learned, did not allow such free reading. More recently, Shavelson 
(1974b) argued that allowing the reader to look back at material while answering 
adjunct questions may lead to a mathemathantic (death of learning) effect. 

This study does not support this well entrenched view. Giving the reader the 
freedom to look back at the passage while answering the low level and high level 
adjunct questions appears to lead to better recall of low level incidental material not 
poorer recall. Perhaps this is because, in their attempts to answer the questions, 
the readers in the FREE group, given the freedom to proceed as they wish, may 
actually read and re-read the passage many times to ensure they are answering 
correctly. 

It is also possible that the fact that a single line was drawn under each question, and 
that the readers may have felt they were expected to write a concise answer on that 
line, caused improved processing of the passage. A further possibility is that the 
adjunct question paradigm was developed during the period when the research 
involved verbatim questions about unconnected sentences, and that it is no longer 
appropriate during the pre^^ent emphasis on higher, or semantic, processing of 
connected discourse. 

The mo^t obvious conclusion from this finding is that adjunct question researchers 
need no longer worry about functioning within such an artificial setting. If the 
results of this finding about presentation sequence are generalizable, distance 
educators, developers of self-sCudy courses, and those instructors who provide their 
learners with texts and readings containing continuous prose.can use questions to 
help their learners to process the prose more appropriately and to facilitate better 
learning of both intentional dnd incidental material. It is also possible that 
Instructors, who h<!vc been using questions to help their students learn, need no 
longer worry that their practice causes a mathemathantic effect. 
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In summary, the FREE groups, or distance education and self-instruction model 
groups, learn as much high level intentional and incidental material as the RQR and 
RRQ adjunct question paradigm groups. Also, the FREE groups learn as much low 
level intentional and incidental material as the RRt groupj and more than the RQR 
groups. This leads this researcher to suggest that the adjunct question 
researchers need no longer restrict their procedures to the traditional adjunct 
question paradigm. They can adopt the self-instruction or distance education model 
and allow the subjects the freedom to proceed as they wish, which includes being 
able to lookback at the passage while answering an adjunct question. 

It is also possible that the findings of adjunct question research to date are 
generalizable to at least distance education, and probably to any real life setting 
where learning from prose is expected. This possibility can only be tested if 
adjunct question research is conducted in more ecologically valid settings, and the 
paradigm altered so that more ecologically sound procedures are followed. 

Level of the Adjunct Questions 

On high level posttest items, readers provided with high level adjunct questions 
were expected to perform better than those readers provided with low level adjunct 
questions. This prediction was based on the results of studies by Watts and 
Anderson (1971) and Friedman and Rickards (1981) which indicate that there is a 
continuum of processing of prose from shallow to deep and that the depth of 
processing of the reader can be manipulated by a parallel continuum from verbatim 
questions through paraphrase and inferential questions to higher level questions 
such as Inference and Implication questions. However, this hypothesis was not 
supported by this study. Readers given low level adjunct questions performed as 
well on high level posttest items as readers provided high level adjunct questions. 

One possible reason for the lack of difference In this study is the different 
definition of low-level questions. Because a high level question was defined as an 
implication or inference question in this study, any question requiring recall, 
whether verbatim or paraphrase or combination of paraphrase and verbatim, was 
defined as low level. This definition is different from that used in typical adjunct 
question research in that only verbatim questions are defined as low level and 
paraphrase questions are defined as high level. 
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Perhaps many of the low level adjunct questions in this study required more than 
verbatim processing and this was enough to cause the deeper semantic processing 
required to find the implications and make the inferences. Friedman and Rickards 
(1981), for example, found that, whil2 both paraphrase and inference adjunct 
questions facilitated deeper processing than verbatim questions, paraphrase and 
Inference questions led to approximately equivalent depth of processing. 

Because distance learners studying for degrees, and those using self-study courses, 
are typically required to process prose materials at a level higher than verbatim 
recall, the definition of high level questions as inference and implication (rather 
than merely paraphrase), and low level adjunct questions as both paraphrase and 
verbatim (rather than merely verbatim) is most appropriate. Depending on the 
educational level of the subjects in an adjunct question study, the subjects typical 
level of functioning will differ. Thus, the definitions of high and low level 
questions should differ with the educational level of the subjects to be appropriate. 
A follow up study of subjects in higher educational institutions should also include 
analysis, synthesis and evaluation adjunct questions. 

Another possible reason for the lack of significant difference in performance on high 
level posttest items between the high and low level adjunct question groups is the 
connectedness of the discourse. The Watts and Anderson (1971) study, for example. 
Investigated the effect of verbatim and paraphrase questions with passages made 
up of independent sentences. The Friedman and Rickards (1981) study investigated 
the effect of verbatim, paraphrase and Inference questions on connected discourse 
about an imaginary country. This investigator, in an attempt to approximate real 
life instructional content, chose connected discourse about a real life problem, the 
likelihood of a future energy crisis. Even though the content was un5'ifiiliar to the 
subjects (most were surprised because they had assumed the opposite was true 
because of the present energy glut) the realness of the content may have caused 
deeper processing ir. those asked low level adjunct questions. 

A further possibility is that che high level adjunct questions were too difficult, 
especially for the grade 10 students scattered among the groups, and thus the 
processlna was no deeper chan that undertaken by those provided with low level 
questions. However, means of between 505< and 60% on the high level posttest 
questions may belie this point. 
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For low level posttest Items It was predicted that the high level adXinct question 
groups would recall as much low level material as those provided with low level 
questions. It was hypothesized that those asked high level adjunct questions would 
actively process the complete passage, inclun .ng the low level material, in their 
attempt to adequately answer the high level adjunct questions. This prediction was 
supported by this study. The high ievel groups recalled as much of the low level 
material as did the low level groups. It is difficult to determine, however, whether 
the lack of difference was due to the above proposition or to the connectedness of 
the discourse. 

The Frequency of Question - Inserted versus Massed 

Andre and Sola (1976) and Andre and Womack (1978) suggest that the amount of 
material between the target text and the related adjunct question determines 
whether or not the adjunct question can be answered effectively so that the 
appropriate processing occurs. Thus it was predicted that adjunct questions 
inserted in the text would facilitate better recall and performance than adjunct 
questions massed in one place. The answer to this question has been sought for 
some time t/ distance educators, faculty developing self-instructional courses and 
textbook writers. While the majority of distance educators and developers of 
self-study courses mass questions at the end of text, as do textbook authors, some 
have been arguing that inserted questions are more effective. 

The results of this study are interesting. The main effect analysis indicates that 
the experimental hypothesis is not supported. There were no significant 
differences between the massed questions groups and the inserted questions groups 
for any of the nine dependent variables. This result is supported by the reviews of 
research to date, in that some studies indicated that inserted questions are 
preferred while others indicate that massed questions are more effective. 

There was an interaction effect that was interesting in that it seemed contrary to 
logic and previous research. Some of the previous studies found that as the amount 
of text between questions increases, the degree of both intentional and incidental 
learning increases for pre-question groups but decreases for post-question groups. 
Others found no interaction. Rickards and DiVesta (1974) found no interaction for 
verbatim level adjunct questions but that higher level adXtnct questions were more 
effecti^ vhen inserted. 
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This investigation found significant interactions for the three high level dependent 
variables and no interactions for the three low level dependent variables. Contrary 
to Rickards and DiVesta, high level adjunct questions we'-e found to facilitate better 
high level performance when massed while luw level adjunct questions facilitated 
better high level performance when inserted. 

This unexpected outconie may in part be explained by the fact that the passages 
were part of connected ^iiiscourse with several themes running through the complete 
text of seventeen passages. Thcst who waited until they had read all 17 passages 
before attempting to answer high level implication and inference adjunct questions 
may have been advantaged by this connectedness of the discourse. 

Another interesting aspect of this interaction is that it occurs only in the P.RQ and 
RQR groups. The FREE groups do »X)t show an interaction. When one remembers 
that the FREE groups were advise'* to follow the same procedures ^fead the 
passage, read the question, re-read the passage, answer the question) whether their 
questions are inserted or massed, then the lack of intr*" action is explained. 

Comparisons with the Control Groups 

The most important result here is that for all incidental dependent variables, most 
treatment gi cups did not score significantly higher (at the 0.05 significance level) 
than the READ-TWICE cc trol group. If this result reflects reality then there is 
one possible conclusion. Adjunct questions do not facilitate the learning of 
incidenta. material any better than does reading th3 passage twice, and distance 
educators and instructional designers can cause better recall of incidental material 
simply by persuading the reader to read the passage twice. 

However, because of the problems with the power of the study, significant 
differences were not found even when treatment group means were over ten 
percentage points higher than ^he control group. If the within group variance had 
been reduced by the choke of a less heterogeneous sample, or if the Duncan test's 
level of significance had been changed from 0.05 to O.iO to conform to the rest of 
thft analysis, then the lack of difference between the READ-TWICE control group 
and treatment groups for incidental material may not have occurred. 
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Suffimary and Cpnclugjgns 

The most important result of this study Is that the FREE groups (which represented 
real-lifu, ecologically sound procedures) did not perform significantly worse than 
the RRQ and RQR groups (which represented the adjunct question paradign) on any 
! the depended ' variables. While this was expected for intentional material, it 
was unexpected for the Incidental material because the forward processing induced 
after reading the question was supposed to be too specific for incidental material to 
be recalled. In t^.ot, the only significant result featuring the FREE group was in 
its favor - the FREE iroup recalled significantly more incidental low order material 
than did the RQR group, and the group mean was greater, although not signif.^cantly 
so, than the RRQ group. 

Thus, this study does not support the well entrenched view that lookbacks cause 
matiiemathantic (death of learning) effects. It is interesting to note that, while 
this view was well known and well adhered to, there were no previous studies which 
compared adjunct question paradigm groups with realistic setting groups. This 
study has shown that adjunct question researchers m^y not need to persist with 
their ecologically invalid methodology. Further, educators who provide their 
learners with questions in the hope that they will facilitate learning can continue 
the practice without undue concern for the views of Shave Ison (1974b), Duchastel 
(1983) and Duchastel and Nungester (1984) that they are doing something terribly 
wrong. 
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